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A HISTORICAL UNDERSTANDING OF HOW SCIENCE WORKS 


Boris Abersek 
University of Maribor, Slovenia 


Introduction 


From Socrates onwards, the Greek cultural life began to take shape, and the whole nature of philosophical and 
scientific thought also began to change. From the natural sciences, interest shifted towards man and his societal 
role. Starting with the Sophists, a critical look at the surviving myths became pivotal at the time. Even if humans 
are still unable to get answers to all the riddles of nature, for which, among other things, mythology offers super- 
natural explanations, we (as human beings) do realize that we are only human. And humans need to learn to live 
together in different forms of social life. The Sophists were interested in man and his place in society. Socrates used 
conversations and tried to help his interlocutors generate valid reasoning and knowledge. According to him, proper 
knowledge must arise inside the individual, which is today’s constructivist approach; no one can instill knowledge 
in the individual from the outside. Only knowledge that comes from within is true understanding. Socrates also 
wanted to find a solid basis for human knowledge. He believed he had found it in human reason. 


From the Past to the Future 


Understanding the processes of science can protect people against misinformation. One of the most intrigu- 
ing moments in science is noticing when something is “weird”. For example, the period between the years 2020 
and 2022 has been a memorable one, unfortunately, for the most part, not in a good way. There was a historic 
disregard for scientific advice regarding the newest case of the pandemic virus. However, while the events of 2020 
may feel unprecedented, the social pattern of rejecting scientific evidence did not suddenly appear in that year. 
There was never any sound scientific reason for rejecting the expert advice on viruses and vaccines, just as there 
has never been any good scientific reason for doubting how humans evolved, that vaccines save lives, or that Al 
and ChatGPT can help people and make our lives better (or worse). To understand the social pattern of rejecting 
scientific findings and expert advice, one needs to look beyond science, towards the past, which reveals that 
many of the various forms of rejecting expert evidence and promoting disinformation are rooted in the history of 
tobacco (Oreskes, 2020). 

Throughout the first half of the 20th century, most people saw science and technology as something that 
makes our lives better. Science has deepened our understanding of the natural world, which has helped us to cure 
diseases; technology lights our homes and brings new forms of entertainment into our lives. Research and develop- 
ment have produced products that measurably improved many lives. However, science was also developing the 
playbook for science denial and disinformation. The chief culprit in this darker story was the tobacco industry. It 
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disparaged science by promoting the idea that the link between tobacco use and lung cancer and other diseases 
was uncertain or incomplete, and that the attempt to regulate it was a threat to human freedom. The tobacco 
industry made products even more addictive by increasing their nicotine content while publicly denying that 
nicotine was addictive. With these tactics, the industry was able to delay effective measures to discourage smoking 
long after the scientific evidence of its harms was clear. The same arguments were used to delay action on acid rain, 
the ozone hole and climate change, and in the year 2020, one could see the spurious “freedom” argument being 
used to disparage mask-wearing or vaccinating. This kind of ‘tobacco strategy’ has also seeped into social media, 
which influences public opinion and which many people feel needs to be subject to greater scrutiny. Computer 
scientists have complained that social media contributes to the spread of Al denial by permitting false or misleading 
claims while hobbling responses by mainstream scientists by labelling their posts as “political”. Without a histori- 
cal perspective, it might seem that this problem was created by a novel technology. Like the tobacco industry, 
social media and IT companies have been selling people a toxic product, while insisting that it was simply giving 
consumers what they wanted. 

With computer-related anxiety, addiction, dependence, stress, and post-traumatic stress disorder on the rise, a 
contingent of mental health professionals are developing a new standard of mental healthcare for our technologi- 
cally changed world. Their profession faces a steep learning curve. There is growing recognition in psychology and 
cognitive science that people are experiencing distress over technological changes. This means that mental health 
impacts could worsen in the future. The human society will have to adapt to many changes, including technological 
ones. People today are dealing with personal problems in the context of such technological changes. Moreover, 
these changes will impact how one deals with personal problems (Kordigel Abersek, Abersek, 2020). 

A bottom-up model should be used to address technological changes, regardless of its illusory appearance 
(Kordigel Abersek, Abersek, 2022). It needs to be clear that the interests of influencers (the IT industry) and the 
interests of the everyman are usually worlds apart. Influential people from IT are aware that they can “buy” their 
world, their own (private) school system, their part of the “paradise”, where they will create “ideal conditions” at 
the expense of others. However, the everyman is the one who will have to actually go to these public schools in 
the (part of the) world that will remain. The strength, the power of the everyman, is in the masses (Abersek, Pesek, 
Flogie, 2023). But the problem is very often a lack of awareness, a lack of faith, and failure to realize that the only 
way to achieve change is to join forces — and that with this in mind, everything that divides us should be forgotten 
and instead, connections should be made between fellow humans, regardless of the differences. Time is running 
out: even in the optimistic scenario, humans are at a deception point, from which there is no return, although pes- 
simists would say that we have already crossed this point a long time ago. People have entered a period of chaos 
and possible great change. The only problem with this is that they need to really know what they want, and to be 
aware that there is only little time left to take action! With all this in mind, it can be said that human beings have 
at least two enemies: 


1. The inner one, ourselves; we (human beings) are (unfortunately) our greatest enemy. 

2. The outer enemy - time, which will keep breathing down our necks and forcing us to make decisions. 
Just think of how the pandemic was (and still is) being addressed: the results are not how we would 
like them to be! The problem, however, is that a permanent solution is possible only in the long term, 
by creating awareness starting with young people throughout the whole vertical of education. 


Conclusion 


If one was to sum up two historical paradoxes, the Moravec paradox' and Simpson's paradox? (Moravec, 1988, 
Simpson 1951), one could conclude that it is not enough simply to “conduct research” and produce statistics using 
all its high-flown statistical methods, without knowing and understanding the problem at hand. One can quickly 
see that what should, by definition, be hard (planning and conducting research or coding) becomes easy, and 
what should be easy (interpreting the results obtained) becomes complicated, and that without understanding 
the problem itself, this can quickly lead to wrong conclusions (Abersek, 2021). 


1 Moravec’'s paradox is the observation in artificial intelligence and robotics that, contrary to traditional assumptions, reasoning requires 
very little computation, but sensorimotor and perception skills require enormous computational resources. 

2 Simpson's Paradox is a statistical phenomenon where an association between two variables in a population emerges, disappears, or 
reverses when the population is divided into subpopulations. 
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To focus only on the technological transformation of today’s society: in light of everything said above, it can 
be said that the term STEAM in the context of digital competence and/or digital literacy does not take into account 
all the components needed for the 21st century (Kordigel Abersek, Abersek, 2020). A possible solution could be 
that educators would approach the challenge of teaching students to engage safely and responsibly with science 
or with technology by focusing on the role of the 4C’s*: 

Critical thinking. It is easy to read an article or see a photo and take it for face value, however, as we all learn from 
“fake news” and social media, this can also imply certain dangers. Students need to learn how to evaluate sources 
and make decisions about how to use the new information. This is of key importance in an age where there is an 
overwhelming amount of information. 

Creativity. Anyone can easily create digital content, but the quality of this content is another question. Teachers 
can help students tap into their strengths and areas of interest while emphasizing original content and developing 
a quality product. To make it relevant and authentic, teachers should encourage students to publish their work 
using a variety of digital tools and elicit feedback in the process. 

Communication. Teachers cannot protect their students from using the internet or Al (for example, ChatGPT), but 
they can help students understand how to generate a positive and safe digital footprint. Communication does not 
take place only face to face or over the phone; it also happens when deciding what to share and whom to share it 
with on the internet. In the age of schools’ massive adaptations to digital learning environments, communicating 
responsibly on the internet has become a vital skill. 

Collaboration. Connecting with other people from diverse backgrounds enhances the ability to understand 
multiple perspectives. Learning from and with others worldwide is possible with digital tools like X, Skype, Padlet, 
ChatGPT, etc. Students can work beyond the confines of their classroom to find answers to real-life problems. Stu- 
dents become better prepared to participate successfully in our global economy through this type of collaboration. 

Including the 4C’s in all applications of technology in the classroom allows teachers to validate what students 
are already interested in, and provides relevant and personalized opportunities for them to grow and learn. It is an 
excellent opportunity for teachers to learn with and from students. Suppose teachers find out what their students’ 
favorite digital tools are and why they like to use them (while stressing how such tools need to be used safely, 
responsibly, and effectively). In that case, these tools can be leveraged within the classroom and used to take 
students outside - even to another country (Kordigel Abersek, Abersek, 2020). 

On the other side, because of the growing importance of technology and engineering (STEAM) in the “digital” 
and Al-oriented educational landscape, proposals of the US National Assessment Governing Board‘ should be taken 
into account. The NAGB decided that technological and engineering literacy would be an essential addition to the 
National Educational Progress (NAGB, 2018). When answering test questions, students are expected to demonstrate 
the following kinds of practices in particular: 


e Understanding Technological (Digital and Al) Principles, 

e Developing Solutions and Achieving Goals, 

¢ Communicating and Collaborating, 

¢ = Teaching/Learning for Citizenship of the 21st Century (Society 5.0), and, finally, 
¢ = Teaching for Living Skills and Competences in Society 5.0. 
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Abstract. The effective use of technology 
in teaching can aid in enhancing the 
teaching and learning of a subject 

matter. This study aimed to enhance 

the technological pedagogical content 
knowledge (TPACK) of five pre-service 
biology teachers (PSBTs) by implementing 
a TPACK-Instructional Design (TPACK-ID) 
model-based technology integration 
course at a Zambian public university. 

An explanatory case study design was 
used. Data sources for the study included 
semi-structured interviews , video lesson 
recordings, lesson plan reports and PSBTs’ 
reflection notes. Data from interviews 

and reflection notes were analyzed using 
deductive thematic analysis while a TPACK 
rubric was used to simultaneously analyze 
lesson plan reports and video lesson 
recordings. The findings indicated that 

the TPACK-ID model-based technology 
integration course improved PSBTs’ overall 
TPACK and sub-knowledge domains: 
pedagogical knowledge, technology 
knowledge, technological content 
knowledge, and technological pedagogical 
knowledge. The study recommends 
implementing context-based technology 
integration courses in teacher preparation 
programs to enhance pre-service teachers’ 
TPACK. 


Keywords: pre-service biology teachers, 
technology integration, micro-teaching 
lesson study, TPACK 
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ENHANCING PRE-SERVICE 
BIOLOGY TEACHERS’ 
TECHNOLOGICAL PEDAGOGICAL 
CONTENT KNOWLEDGE 
THROUGH A TPACK-BASED 
TECHNOLOGY INTEGRATION 
COURSE 


Anthony Bwalya, 
Marcellin Rutegwa, 
Dorothy Tukahabwa, 
Thumah Mapulanga 


Introduction 
Background 


The advent of digital technologies has dramatically changed the way 
teaching and learning are conducted in schools today. Bueno et al. (2023) 
asserted that if digital technologies have transformed society, then educa- 
tion and teaching practices must evolve with it. The skills and knowledge 
needed for teachers to effectively work in the 21% century are changing, 
and so should the education provided to them (Mukuka et al., 2023). Conse- 
quently, education researchers worldwide are concerned about developing 
teachers’ knowledge and skills in technology integration for teaching (Bwalya 
& Rutegwa, 2023; Mbwile & Ntivuguruzwa, 2023). In recent years, the techno- 
logical pedagogical content knowledge (TPACK) framework (Mishra & Koehler, 
2006) has been used to guide the development of teachers’ competencies in 
teaching with technology, particularly in science and mathematics teaching. 
TPACK is the competency required for teachers to teach a specific subject 
matter competently with technology. TPACK has since been recognized 
as an essential requirement for 21% century teachers, as it allows them to 
integrate content, pedagogy, and technology in their teaching practices. To 
this end, several researchers have underscored the significance of preparing 
pre-service teachers (PSTs) for technology integration during their initial 
teacher training program (Durdu & Dag, 2017; Kafyulilo et al., 2016; Tondeur 
et al., 2013, 2020). 

Despite the efforts to prepare PSTs for teaching with technology, there 
is aresounding consensus among researchers that PSTs struggle to integrate 
technology into their teaching (Agyei & Voogt, 2012, 2012; Aktas & Ozmen, 
2020a; Cetin-Dindar et al., 2018; Jita & Sintema, 2022a; Meng & Sam, 2013; 
Mouza, 2016; Pondee et al., 2021). This problem has partly been attributed to 
a lack of training on technology integration during teacher training (Agyei & 
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Voogt, 2012; Alayyar et al., 2012; Jita & Sintema, 2022b). Teacher training institutions (TTls) must provide PSTs with 
the knowledge, skills, and experience needed to effectively integrate technology into their practice, considering 
the interplay between pedagogy, content, and technology. Providing PSTs with training on technology integration 
is a good way for PSTs to get acquainted with competencies for technology integration in their classroom practice. 

However, most TTls in sub-Saharan Africa do not offer courses on integrating technology in teaching and 
learning, thus leaving PSTs inadequately prepared for technology integration in their classroom practice (Agyei & 
Voogt, 2012; Jita & Sintema, 2022b). A baseline study conducted by Bwalya and Rutegwa (2023) revealed that PSTs 
enrolled at two TTls in Zambia had low to moderate TPACK self-efficacy. This suggested that PSBTs were hardly 
prepared for technology integration in their classroom practice. This background information served as a founda- 
tion and motivation for the current study. This study aimed to fill this gap by implementing a TPACK-Instructional 
Design model-based technology integration course to enhance pre-service biology teachers’ TPACK at a public 
university in Zambia. Although another study was conducted by Bwalya et al., (2024) to develop TPACK of PSBTs, 
this current study aimed to delve more into the process of PSBTs’ TPACK development. The methods and analytical 
approaches for the two studies are different. Bwalya et al. (2024) used a mixed methods approach while the current 
study utilizes the explanatory case study design to get a more in-depth description of PSBTs’ TPACK development. 


Theoretical Framework 


This study used Koehler and Mishra’s (2009) TPACK integrative framework as a theoretical and analytical tool 
to enhance pre-service biology teachers’ (PSBTs’) TPACK. The TPACK framework is made up of seven knowledge 
domains namely; technological knowledge (TK), content knowledge (Ck), pedagogical knowledge (PK), techno- 
logical pedagogical knowledge (TPK), technological content knowledge (TCK) and technological pedagogical 
content knowledge (TPACK). Below is a description of how each of these knowledge domains is applied in the 
context of this study: 

1. Technological Knowledge (TK): This refers to the PSBTs’ knowledge and skills needed to use technology 
effectively in teaching. It includes using digital tools for virtual simulations, and laboratory experiments, 
online and collaborative resources for teaching and learning biology. 

2. Pedagogical Knowledge (PK): This term refers to the knowledge and skills that PSBTs need to design and 
implement effective teaching and learning strategies for biology. It is not limited to the use of strategies 
such as inquiry-based learning, project-based learning, and teacher-led demonstrations, rather it also 
includes other active learning strategies that engage students in the scientific process. 

3. Content Knowledge (CK): This refers to the PSBTs’ knowledge needed to teach secondary school biol- 
ogy effectively. This involves not only a deep understanding of core concepts in biology, but also an 
awareness of emerging trends in the field of biology. 

4. Technological Pedagogical Knowledge (TPK): Represents the PSBTs’ knowledge and skills needed to 
design and implement effective technology-enhanced learning activities in a biology classroom. This 
includes creating and using multimedia and online resources (such as YouTube videos, PPTs) to illustrate 
complex biological concepts. 

5. Technological Content Knowledge (TCK): Represents PSBTs’ competency in manipulating technologi- 
cal devices to aid the teaching and learning of biology content. This includes using online resources 
to access biology content, effective tools for teaching specific biology concepts to enhance learners’ 
understanding. 

6. Pedagogical Content Knowledge (PCK): This represents PSBTs’ knowledge of teaching approaches suitable 
for effectively presenting biology concepts in different learning environments. It also includes PSBTs’ 
knowledge of classroom management and assessment techniques. 

7. Technological Pedagogical Content Knowledge (TPACK): Entails PSBTs’ knowledge and skills in selecting 
appropriate technologies for representing specific biology concepts and using appropriate strategies 
to enhance learners’ understanding of the biology concepts being represented. 

TPACK framework is particularly relevant for teaching biology, given the rapidly evolving nature of the field 
and the increasing importance of technology in teaching and learning. In the context of this study, the TPACK 
framework emphasizes the integration of three types of knowledge: technological knowledge (TK), pedagogical 
knowledge (PK), and content knowledge (CK). 
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Conceptual Framework 


The conceptual framework was grounded in the situated learning theory. This theory asserts that teachers 
learn when they are actively engaged in their own learning in contexts that are relevant to them (Greene & Jones, 
2020; Voogt et al., 2016). The situated learning theory further stresses that students acquire knowledge by ob- 
serving others and engaging in practical activities, thus becoming cognitive apprentices (Kurt, 2021). The theory 
emphasizes the role of experts in supporting learning by scaffolding the learning process of apprentices through 
constant interactions. In the context of this study, PSTs will enhance their TPACK competencies through constant 
interactions with the experts in technology and biology education as well as gaining practical experiences in 
demonstrating technology-rich lessons in micro-teaching lesson study. These interactions occur in a context that 
is meaningful to the PSBTs. The PSBTs are given practical teaching and learning experiences with technology by 
guiding them to design and demonstrate biology lessons in a technology-rich environment. When this occurs, 
PSBTs learn and gain enough competencies to become cognitive apprentices. 


Literature Review 
TPACK Instructional Design Models to Guide the Development of Teachers’ TPACK 


Using TPACK-Instructional Design (TPACK-ID) models to improve PSTs TPACK has become common. Several 
researchers (Jang, 2010; Jang & Chen, 2010; Lee & Kim, 2014a, 2014b, 2017a; Tondeur et al., 2012) have made at- 
tempts to develop teachers’ TPACK by using TPACK-ID models. 

Jang (2010) constructed the TPACK-COIR (Comprehension, Observation, Instruction and reflection) model 
after examining data from in-service science teachers’ assignments, reflective journals and interviews. This model 
proposed the use of interactive whiteboard (IWB) technology and peer coaching to develop the TPACK of science 
teachers in an authentic classroom setup. The TPACK-COIR has four main activities which include: 1) Comprehen- 
sion of TPACK, 2) Observation of peer teaching, 3) Teaching in a real classroom, and 4) Reflection on TPACK. The 
study found that the TPACK-COIR model is effective in developing science teachers’ TPACK (Jang, 2010). Tondeur et 
al. (2012) designed the synthesis of qualitative evidence (SQD) model for developing PSTs to integrate technology 
into their teaching. The SQD model proposed six key themes after evaluating studies that implemented various 
strategies to prepare PSTs for technology integration in classroom practice. The six key themes of the SQD model 
include: 1) teacher educators as role models, 2) Reflecting on the use of technology in teaching, 3) learning the use 
of technology by design, 4) peer collaboration, 5) scaffolding meaningful technology practices, and 6) feedback. 
The SQD identified critical factors that need to be considered when designing training to prepare pre-service 
teachers for technology integration. Firstly, observing a teacher using technology in their teaching was identified 
as a key motivator for PSTs to integrate technology into their own practice. Secondly, PSTs need to be afforded the 
opportunity to collaboratively plan lessons that incorporate technology in their specific content areas. Addition- 
ally, PSTs should be able to experience the pedagogical integration of technology in the classroom during their 
training. Lee and Kim, (2014a, 2014b, and 2017) have developed three models focused on preparing PSTs’ TPACK 
development. The first TPACK model consists of six stages: introduce, demonstrate, develop, implement, reflect 
and revise (IDDIRR). During the introduce stage, the instructor introduced the TPACK framework. This was aimed 
at making the PSTs become aware of TPACK and its sub-knowledge domains. The Demonstrate stage involved an 
instructor demonstrating a technology-rich lesson. The develop stage involved pre-service teachers in groups 
to design TPACK-based lessons. The implement stage involved one representative group member to present the 
lesson, while the reflect stage involved respective group members reflecting on the presented lesson, discussing 
its strengths and weaknesses with a view of improving it and the revise stage involved members collaboratively 
revising the lesson after deep reflection and discussing the pros and cons of the previous lesson. The TPACK-IDDIRR 
study reported improvements in PSTs’technological knowledge, pedagogical knowledge and content knowledge, 
while no clear evidence was found in the PSTs’ TPK, TCK and TPACK domains (Lee & Kim, 2014a). The second ver- 
sion of the TPACK-IDDIRR Lee and Kim, (2014b) was then formulated after revising the initial model. The second 
model emphasized PSTs’ pedagogical-related knowledge as being key to developing TPACK. The TPACK-IDDIRR2 
consisted of three key stages: 1) Understanding TPACK, 2) Engaging in TPACK, and 3) Practicing TPACK. Results from 
the second TPACK-IDDIRR model revealed that stages 2 and 3 of the model improved PSTs’TPACK. Version Ill of the 
TPACK-IDDIRR model (Lee & Kim, 2017a) integrated the distinctive, integrative and transformative views of TPACK 
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with three key phases: 1) Understand TPACK, 2) Experience TPACK, and 3) Practice TPACK. Results from version III 
of the TPACK-IDDIRR model effectively enhanced PSTs’ TPACK. Among the strategies used, role-playing was found 
to be the main contributing strategy to enhancing PSTs’ understanding of TPACK. 

The ID models reviewed helped to synthesize the key elements required to provide systematic teaching pro- 
cedures in a technology integration course. From the reviewed studies, key stages were identified which informed 
the adaptation of the model used in this study. The studies reviewed suggest that TPACK-ID models are an effective 
way of developing PSTs’ TPACK. However, few studies have implemented the TPACK-ID model-based technology 
integration course in a biology context and conducted in the African context to develop TPACK of PSTs Jang & 
Chen, 2010; Jimoyiannis, 2010; Lee & Kim, 2014b) 


Micro-teaching Lesson Study 


The combination of microteaching with lesson study is known as micro-teaching lesson study (MLS). MLS 
brings together a group of teachers to collaborate on the design of a lesson created for a specific student-learning 
objective (Aktas & Ozmen, 2022). It consists of numerous steps, including collaborative lesson preparation, lesson 
observation by colleagues and other expert advisers, analytic reflection, and continuous revision. When employed 
in a teacher education program, the MLS process often includes pre-service teachers creating a lesson plan and then 
presenting that lesson to a small group of peers (Meng & Sam, 2013). The peer teachers being taught are expected 
to critique the lesson and provide constructive feedback which can be used to improve the lesson. MLS has been 
used in several studies to improve the TPACK of pre-service science and mathematics teachers. Studies by Aktas and 
Ozmen, (2020, 2022) and Pondee et al. (2021a) that used MLS have shown that MLS is effective in enhancing PSTs 
TPACK as they provide a platform for PSTs to demonstrate their knowledge and skills in teaching with technology. 


Problem Statement 


Several studies (Bwalya & Rutegwa, 2023; Durdu & Dag, 2017; Garcia et al., 2021; Pondee et al., 2021a; Tho- 
hir, 2023; Umutlu, 2022) have highlighted that pre-service teachers face difficulties of technology integration 
into their classroom practice. Additionally, there is a dearth of research focusing on PSTs’ TPACK development in 
specific subject contexts. Most of these studies mentioned above focused on science in general or mathematics. 
None specifically focused on the development of PSBTs’ TPACK. Further, none of these studies was conducted in 
the Zambian context. Therefore, the current study sought to close this gap by developing PSBTs’ TPACK through a 
TPACK-ID model-based technology integration course at a public university in Zambia. 


Research Aim and Research Questions 


This study aimed to enhance the technological pedagogical content knowledge of PSBTs using the TPACK-ID 
model-based technology integration course. The study contributes knowledge to fill the gap by answering the 
following research questions; 

1. How does the TPACK-ID model-based technology integration course influence the TPACK development 
of PSBTs? 

2. How do the PSBTS’s TPACK and sub-knowledge domains change during the micro-teaching phases of 
the technology integration course? 


Significance 


The study provides useful information on the strategies and conditions necessary for developing pre-service 
teachers in a specific subject (biology) context. The study also highlights key stages in a TPACK-ID model-based 
course useful for developing pre-service teachers’ TPACK. This study identifies the need for designing and imple- 
menting TPACK-ID model-based technology integration course at teacher training institutions to enhance pre- 
service teachers’ TPACK. 
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Research Methodology 

Design 


This study used the explanatory case study design to get an in-depth qualitative description of PSBTs’ TPACK 
development. A case study is defined as an in-depth description and analysis of a bounded system (Creswell, 2014). 
A case study helps to obtain in-depth information regarding an event, subject or a phenomenon under study. This 
study describes five pre-service biology teachers’ bounded case and how their TPACK evolved after participating 
in the TPACK-ID model-based technology integration course. 


Participants and Context 


The study involved 50 fourth-year PSBTs (28 females and 22 males), between the ages of 18 and 34, who 
were enrolled in the Department of Life Sciences at a Zambian public university during the academic year 2021- 
2022. Five (5) model PSBTs were conveniently selected from the 50 PSBTs enrolled in the technology integration 
course to provide more in-depth data. The PSBTs had completed various biology courses such as Introduction to 
Cell and Molecular Biology, Genetics, Advanced Physiology, Plant Taxonomy and Evolution, and General Ecology. 
Additionally, they had already taken Science teaching methods, a general pedagogy course, and were currently 
enrolled in a content-specific course for teaching biology - Biology teaching methods. The participants also had 
some practical teaching experience, having completed a three-month teaching practice in a secondary school 
environment. The participants were in their final year of study and were assumed to have gained enough content 
and pedagogical knowledge to provide rich data for the study. Before participating in the study, all participants 
provided their informed consent. 


Research Instruments 
Semi-structured Interview Schedule 


The semi-structured interview schedule was formulated by the researchers after an extensive literature search 
(Aktas & Ozmen, 2020a, 2022; Cetin-Dindar et al., 2018). The interview schedule was then subjected to expert review 
and validation by three biology education specialists who commented on the clarity, relevance, and sufficiency of 
the questionnaire items. Their comments were used to improve its reliability and validity. The sample questions of 
the interview schedule are shown in Appendix 1. 


Data Sources and Procedures 


The main data sources used in the study include; semi-structured interviews, video recordings of lessons, 
lesson plan reports, and PSBTs’ reflection notes. 


Semi-structured Interviews 


Five PSBTs (3 males and 2 females): Andre, Mugabo, Seraphine, Venuste and Wivine (pseudo names) con- 
veniently selected from the 50 PSBTs enrolled in the course, were interviewed twice, (before the course began 
(Pre-test) and after it ended (Post-test). While some questions from the first interview were repeated in the sec- 
ond interview, others were only asked at the start. The interview duration averaged around 25 minutes and was 
recorded, transcribed, and analyzed. 


Lesson Plan Reports 
Before the course, five conveniently selected PSBTs were requested to create lesson plans incorporating tech- 
nology. During the training process, they were provided with an example of a 5E lesson plan model, which they 


utilized in their respective groups during collaborative lesson planning for micro-teaching. The PSBTs were permit- 
ted to discuss the presented lesson in their groups and provide reflections. Following this, they were instructed 
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to re-plan the same lesson using the same model (5E), incorporating the points raised during the discussions and 
their individual reflections on the previous lesson. Each of the five groups designed a total of three lesson plans, 
resulting in a total of fifteen (15) lesson plans. The 5E lesson plan model was used since it is effective and engages 
students. All the lesson plans were collected and carefully analyzed. 


PSBTs’ Reflection Notes. 


At the end of each week, the PSBTs were instructed to write reflective notes regarding their experiences dur- 
ing the course. The purpose of these notes was to encourage the PSBTs to comment on what they had learned, 
the challenges they had encountered, and any potential improvements that could be made to overcome these 
challenges in the future. 


Video Recordings 


To assess the TPACK implementation skills of the PSBTs, the researcher used video cameras to record their 
lesson presentations. This enabled both the researcher and co-researcher to observe and analyze the presentations 
in detail. The PSBTs were also given access to the recorded videos to evaluate their performance and reflect on 
their teaching strategies. Following extensive discussions with the researcher and among themselves, the PSBTs 
were allowed to re-plan and deliver a second and third presentation. These presentations were also recorded and 
lasted for 40 minutes each. 


Course Implementation Following the TPACK-ID Model 


The study used a technology integration course which applied the TPACK-ID model to develop PSBTs TPACK. 
The intervention lasted for 6 weeks, and the class met twice weekly, each session lasting for two hours. The par- 
ticipants were also given tasks to do outside the class hours. 

The course followed the three key elements and stages picked after evaluating the reviewed studies that 
implemented various TPACK-ID models. The three key themes which came out were: 1) introducing and comprehen- 
sion of TPACK, 2) Demonstrating and Observing TPACK (Teacher educators as role models while students observe), 
and 3) Practicing TPACK (learning TPACK by design and practice). During the first stage (introducing and comprehen- 
sion of TPACK), the instructor herein, called the researcher, introduced the TPACK concept and explained the seven 
TPACK knowledge domains and their interactions. This was aimed at making the PSBTs aware of TPACK and its sub 
knowledge domains needed for successful technology integration in teaching. The PSBTs were also allowed to 
discuss the various TPACK domains and give practical examples. This was meant to make the PSBTs understand 
or comprehend TPACK. 

The second stage was the Demonstration and observation stage. During this stage, the PSBTs were introduced 
to different education technologies relevant for teaching and learning biology. The PSBTs were also introduced to 
various teaching strategies useful for effective teaching with technology. The technologies which were taught in 
the course included presentation tools (PowerPoint), interactive virtual reality tools (PhET Simulations, animations, 
and Virtual labs), Web 2.0 tools (internet, WhatsApp, YouTube, and Google Forms) and hardware tools (Projector and 
smart boards). An ICT expert was tasked with demonstrating how different technological tools can be operated 
while the researcher discussed their use for teaching and learning biology. The researcher demonstrated a biology 
lesson in a technology-rich environment during this stage. The researcher demonstrated the concept of natural 
selection using PhET simulations using a guided inquiry approach. The PSBTs observed the lesson. The researcher 
was acting as a role model to demonstrate the effective use of technology in teaching. 

The third stage involved Practicing TPACK or learning TPACK by design and practice. PSBTs were engaged in 
micro-teaching lesson study which allowed them to collaboratively design technology-rich lessons, demonstrating 
the lesson, reflecting on the presented lesson, and revising the lesson. At this stage, the PSBTs were divided into 
groups of ten and tasked to choose a topic, collaboratively design a lesson, and choose the appropriate technol- 
ogy and teaching approach for presenting the lesson. The designed lesson was then implemented or taught by 
one model teacher (representative group member), while the researcher and other group members observed. The 
lesson was videotaped and later given to the presenter and respective group members to allow them to reflect 
on the lesson so as to make improvements. The researcher also provided feedback on the lesson presented. The 
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PSBTs in their respective groups reflected on the presented lesson, discussing its strengths and weaknesses to 
improve it and then collaboratively revised the lesson after deep reflection and discussing the pros and cons of the 
previous lesson. The process of presenting, reflecting, and revising followed the iterative learning process and thus 
was done repetitively 3 times. Table 1 shows a summary of the implemented TPACK-ID model-based technology 
integration course with the key themes. 


Table 1 
Implemented Technology Integration Course Based on the TPACK-ID Model 


Stage 


1 Introduce and dis- 
cuss TPACK (TPACK 
comprehension) 


Module Outcomes 


Content/Activity 


Objective/s 


Introduction e Introducing the TPACK course, facilitators and know- | TPACK awareness 
ing the participants 
¢ Getting the expectations from the participants 
TPACK concept. ¢ TPACK framework Understanding TPACK 


e Explaining the TRACK framework 


¢ Discussion on the TPACK framework and its implica- 
tions for teaching and learning of biology 


2 Demonstrate and Enhancing TPK, TCK, PK 
Observe TPACK 
(Teacher educator as 


role model) 


Use of digital technol- 
ogy to Support 
Pedagogy 


The importance of using digital technologies in teach- 
ing and learning; 

(a) Mobile devices; (b) Computer with accessories; (c) 
Projector; (d); interactive whiteboards 


Enhancing CK, PK, TCK, PCK, 
TPK, TPACK 


e Relevant digital tools & resources: (a) Digital 
instructional materials Research-based science 
simulation and Animations available at (https://phet. 
colorado.edu/), https:/Awww.labxchange.org/, Pow- 
erPoint Presentations, google forms, discussions of 
their effective use. 

1.Guided inquiry 

2.Teacher-led demonstrations 

3.Class discussions 


Introduction and 
use of instructional 
technologies 


Teaching strategies 
useful for effec- 
tive teaching with 
technology. 


Lesson demonstra- ¢ Presentation of sample lessons in specific biology | Enhancing CK, PK, TK, TCK, 


tion topics. Use appropriate technology to provide a solid | TPK,PCK, TPACK 
example and act as a role model in teaching biology 
with technology 
3, 4 and 5 Practice Micro-teaching les- ¢ PSBTs to prepare lessons that integrate technology, © Enhancing CK, TK, PK, TCK, 


TPACK son study: Peer les- and present to their peers. PSBTs to reflect and = TPK, PCK, TPACK 

(Develop, Implement, — son demonstrations discuss the lesson in groups for possible improvement 

Reflect & revise) and discussions 
Preparing and ¢ PSBTs in groups to re-plan lessons and teach their © Enhancing CK, TK, PK, TCK, 
teaching lessons with peers. PSBTs reflect, re-plan and present the lesson TPK, PCK, TPACK 
technologies 


Data Analysis 
Semi-structured Interviews 


The researchers used deductive thematic analysis by creating themes based on the TPACK framework to 
code the selected excerpts from the semi-structured interviews. The interviews with PSBTs were recorded and 
later transcribed. The interview transcription process immersed the researchers in the data and allowed them to 
think about what the respondents were saying in order to come up with themes. The transcripts of the interviews 
were read repeatedly by authors 1 and 4 to come up with appropriate themes representing the different TPACK 
domains. The appropriate excerpts that describe the PSBTs’ TPACK competencies were identified and reported. 
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Data from the lesson plan reports and video recordings was analyzed simultaneously using the TPACK rubric 
designed by the researcher. Three biology education lecturers reviewed the TPACK rubric to determine its validity 
and their comments were used to improve the clarity and consistency of the rubric. The TPACK rubric consisted of 
four performance levels extending from limited to expert. The maximum score that one could get was 4, repre- 
senting expert level, with 3 representing advanced level, 2 basic, and 1 representing limited level. Two researchers 
independently scored the video lesson presentations and lesson plan reports following the rubric criteria. The 
two researchers then came together to compare their scores to avoid biases and ensure data validity. Where the 
researchers scored differently, they discussed and reached a consensus. The results from the lesson plan reports 
and video lesson recordings were triangulated with results from interviews to ensure their validity. Part of the 


TPACK rubric is shown in Table 2. 


Table 2 


The TPACK Rubric for PSBTs 


Level of knowledge 


TPACK 
Domain aera 
Limited (1) Basic (2) Advanced (3) Expert (4) 
CK Understanding of Biology The PSBT explains The PSBT explains The PSBT explains biology The PSBT explains biology 
concepts biology concepts biology concepts with concepts without difficulty concepts without difficulty, 
with difficulty andno some difficulty and uses and uses only one exam- _ uses more than one exam- 
examples only one example per __ ple per concept ple per concept 
concept 
PK Use of learner prior The PSBT does not The PSBT determines The PSBT determines The PSBT determines the 
knowledge determine learners’ the learner prior learners’ prior knowledge, __ learners’ prior knowledge 
prior knowledge knowledge but does and comments on it and builds from it 
not connect it with the 
lesson 
TK Knowledge of hardware The PSBT does not The PSBT operates the The PSBT operates, The PSBT operates, 
and software relevant for use(setupanduse) technology withsome —_ navigates, and commands _ navigates, and commands 
teaching biology any technology at all _—_ difficulty the technology with some __ the technology confidently 
confidence and with comfort 
TPK Selection of technology The PSBT did not The PSBT selected The PSBT selected The PSBT selected 
that suits teaching strategy select any technology technology that does technology that loosely technology and teaching 


for the lesson 


not match the teaching 
strategies 


matches the teaching 
strategies 


strategies that match well 


TCK Selection technology that +The PSBT did notse- ThePSBTselecteda | The PSBT selected The PSBT selected 
suits content being taught lect any technological technological tool inap- _ technology that loosely technology appropriate for 
tool for the content. propriate for the content matches the content the content 
PCK Illustrations to address The PSBT uses limited The PSBT illustrates the The PSBT illustrates the | The PSBT illustrates the 
misconceptions illustrations such that — biology concept(s) with content using challenging —_ content using challenging 
the biology concept(s) some relevant examples and relevant examples and relevant examples that 
being taught is/are not that are only useful in with no focus on likely take care of anticipated 
Clarified clarifying concepts but + misconceptions misconceptions 
not challenging 
TPACK The interaction of knowl- = The PSBT does not The PSBT uses technol- The PSBT uses relevant The PSBT uses relevant 
edge domains for effective use technology to sup- ogy in ways that do not technology to support the — technology to support the 
teaching of biology port biology teaching — support content delivery delivery of the content learner's exploration of the 
and learning or exploration content 
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To ensure credibility, the authors spent a significant amount of time analyzing data. Researchers 1 and 4 con- 
ducted the initial analysis, which was confirmed by researchers 2 and 3. To ensure dependability, detailed descrip- 
tions of data collection and analysis procedures were provided. The TPACK framework was used to frame the study 
and guide the analysis procedures to ensure conformability. In addition, interview excerpts were used to report the 
findings. The research instruments were thoroughly described. An in-depth description of the participants and the 
study's context was described for transferability. Furthermore, multiple data sources such as interview data, lesson 
plan reports, video lesson recordings and PSBTs’ reflection notes were used to triangulate the findings, thereby 
ensuring the reliability and validity of the study. Data triangulation is the use of various data sources to validate 
emerging findings (Durdu & Dag, 2017). 


Research Results 
Results from Lesson Plan Reports, Video Lesson Analysis, Interviews and Reflection Notes 


Our analysis aimed to identify evidence of TK, TPK, TCK, and TPACK in the PSBTs' lesson plan reports and video 
lesson recordings. We triangulated the findings with data from interviews and reflection notes to gain a deeper 
understanding of the PSBTs’ TPACK competencies and how the TPACK-ID model-based technology integration 
course influenced them. 

The data obtained from lesson plan reports and video lesson analysis indicate that during the first lesson 
presentation, PSBTs had a basic level of TK. Before course implementation, the most commonly used technology for 
teaching biology was PowerPoint presentations (60%) and YouTube videos (40%). Although the PSBTs identified a 
projector as a useful technological tool for teaching, they struggled to connect the projector to their laptop computer. 
In most cases, they were helped by either the researcher or peer group members to connect their laptops to the 
projector and project their presentation. For example, Seraphine pointed out in her weekly reflection note that she 
had no knowledge of using the projector during her first lesson presentation, which made her nervous, she wrote: 


| was very nervous during my first lesson presentation because | did not know how to connect the projector to the 
laptop. But, | became more confident in my second lesson because | became familiar with the projector and how to 
use it after being trained and also after seeing what my friends were doing. 


Similarly, when Venuste was asked about technologies that could be used for teaching and learning biology 
during the pre-test interview, he stated that he had little knowledge about it and only knew how to use Power- 
Point. He had this to say: 


| can't say | have much knowledge about it, | have little knowledge about it. | only know how to use PowerPoint. It is 
better to use PowerPoint when teaching biology because it is bulky so when it comes to writing notes on the board 
it can be challenging, but if you have a PowerPoint, it can simplify your work. 


This clearly showed a lack of experience in teaching with technology. The PSBTs’TK was at a basic level (100%) 
during their first lesson presentation. However, after the course training, the PSBTs’ TK improved to an advanced 
level (80%) while 20% was at a basic level. The PSBTs became acquainted with various technologies for teaching 
and learning biology, including PhET simulations, virtual laboratories, Google assessment forms, animations, You- 
Tube videos, and PowerPoint presentations. When asked about what he had gained from the course, Venuste said: 


| have learnt how to operate technological devices, for example, how to connect the computer to the projector, and 
also how to create a PowerPoint presentation, how to design online quizzes on the Google platform and share them 
with the class, and how to use simulations like PhET. 


Another PSBT pointed out that the training on different technologies and having the chance to demonstrate the 


lesson during the technology integration course helped her to learn how to create and use PPTs, Google forms and 
PhET simulations. Wivine had this to say about the TK competencies gained from the Technology integration course: 
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lam now able to create PowerPoint presentations and now familiar with online assessment tools like Google Forms | 
can use for assessing my teaching. | am also able to use PhET simulations which can help in teaching abstract concepts 
in biology with ease. The training on different technologies helped me to become familiar with them. Also having the 
opportunity to use them in my lesson presentation helped me to become confident in using them. 


In terms of their TPK, the PSBTs demonstrated a basic level (80%), Limited level (10%) and advanced level (10%) 
before course implementation. The PSBTs were able to select a technology for use, but they could not match it ap- 
propriately with a teaching strategy. For example, some of the PSBTs who mostly used the teacher's exposition (80%) 
and question and answer method (60%) could not properly utilize PowerPoint presentations. Venuste, for instance, 
did not properly engage his learners in the first lesson as he spent most of the time reading and explaining what 
was projected on the screen instead of engaging his learners in the lesson. During the pre-test interview, PSBTs 
were asked if they could identify a specific teaching strategy that would work well with a specific technological 
tool. Venuste responded, “No, not really. What | know is that you can use a computer for many different teaching 
strategies.” Similarly, Andre acknowledged his ignorance by stating that, “| am not really sure” However, Mugabo 
was able to correctly match the technological tool with a teaching strategy. When responding to the question, he 
said,“| can use a microscope for practical demonstration of magnification.’ Overall, the PSBTs demonstrated a basic 
level of TPK before course implementation. However, after the course implementation, lesson plan reports and 
video lesson analysis results indicated that PSBTs’ TPK was at an advanced level (90%) and basic level (10%). PSBTs 
demonstrated improved competency in matching the technology and teaching strategy. Venuste, for example, in his 
second and third lesson decided to use the animation available at a free website which the PSBTs were introduced 
to during the training available at https://www.labxchange.org/library/items/Ib:LabXchange:7211c006:video:1 with 
teacher-led demonstration to teach DNA replication. Similarly, Mugabo demonstrated improved TPK and TCK by 
creating and sharing an assessment item on digestion using Google Forms. The link to the assessment was shared 
using a WhatsApp group created by PSBTs for the technology integration course. The PSBTs and the instructors 
were then asked to view the form and give feedback. The Google form assessment item created can be retrieved 
using the link: https://www.docs.google.com/forms/d/1o0cRYeAfFLTKD6qbkXWhhiLmyHp9Ja0tD7eYSm28K6g4/ 
edit. Part of the Google form assessment item is shown in Figure 1. 


Figure 1 
Part of the Google Form Quiz Created by Mugabo 
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The PSBTs showed a basic understanding of TCK before course implementation. The PSBTs did not recog- 
nize that different topics in biology would require a different technology for effective presentation or learning. 
During their first lesson presentation, most PSBTs did not consider the topic they were teaching when selecting 
a technological tool to use. For instance, Wivine, who was teaching diffusion, decided to show a YouTube video 
using her phone to demonstrate the diffusion of potassium magnet crystals. When Seraphine was asked during 
the pre-test interviews if she thought that the use of technological tools would be different in different topics of 
biology, she replied, “No, not really. | think they would be the same. PowerPoint can be used to teach all topics in 
biology”. However, after course implementation, Seraphine had a different response to the same question. Dur- 
ing the post-test interviews, she said, “Yes, because every topic is different, in some topics, maybe you might use 
power points, while in some a simulation would be useful. When reflecting on the weekly course, Andre noted: 


This week’s training has helped me to select technology which suits the topic | am teaching. The discussions with my 
friends helped me to see that different topics require different modes of presentation in order to make it easy for the 
understanding of the learners. 


Results from video lesson analysis and lesson plan reports indicate that the TPACK of PSBTs was at a limited 
level (60 %) and basic level (40%) before course implementation. The PSBTs were unable to properly select a tech- 
nological tool to match the topic and support the teaching strategy being used. In most cases, the PSBTs were 
only using technology because it was what they knew. Furthermore, PSBTs’ limited TPACK was evidenced by the 
response to question item 8 in the interview guide. The question was asked during the pre-test interviews and 
post-test interviews. Some of the responses given during the pre-test interviews are as follows: 


“No, |am not able to identify”- Wivine (Pre-test Interviews) 

“No, not really. | think that maybe | can use PowerPoint to show muscle structure”-Andre (Pre-test Interviews) 

“lam not really sure, maybe on DNA replication | will use PowerPoint to show how DNA is replicated’- Seraphine 
(Pre-test Interviews) 


Venuste, on the other hand, demonstrated a basic understanding of TPACK before course implementation 
as evidenced by his response. He said: 


Yes, on respiration, when you are teaching on respiration, for example, two types of respiration are taking in and taking 
out of the air, inspiration, and expiration. We do explain how the ribs move inwards and outwards, so on that topic, 
if | can use PowerPoint to show how the ribs and the diaphragm move it will help the learners to see what happens 
during inspiration and what happens during expiration. 


However, after the course implementation, results indicated that PSBTs had advanced TPACK level (60%) and 
basic TPACK level (40%). Their responses to the interview question for the TPACK domain also showed improved 
TPACK. Some of the responses in the post-test interviews are as follows: 


Yes, | can talk of locomotion where you are explaining about arm bone movements and antagonist action of muscles 
then you can use teacher-led demonstration to show the action of muscles and bone movements- Andre (Post-test 
Interviews) 

Yes, if | want to teach diffusion, | can use Simulations or Virtual laboratories using a guided inquiry approach. - Wivine 
(Post-test Interviews) 


Wivine effectively used guided inquiry to guide her learners through a simulation on diffusion which is found 
at https://phet.colorado.edu/sims/html/diffusion/latest/diffusion_en.html. Wivine explained clearly the purpose 
of using the simulation and explained to her learners how to manipulate the simulations in order to see the dif- 
ferent results. She encouraged them to make conclusions based on the observations made. The interface to the 
simulation used to teach diffusion is shown in Figure 2. 
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Figure 2 
Interface for Diffusion PhET Simulation Used to Teach Diffusion 
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The following section presents findings from the PSBTs’ microteaching activities. The remarks represent 
the consensus reached by the researchers after a thorough analysis of the lesson plan reports and video lesson 
recordings of the lessons presented by the representative PSBTs. Table 3 shows the findings from selected TPACK 


domains of interest. 


Table 3 


Microteaching Groups, Lessons Taught and Teaching Strategy Used, and Technological Tool (s) Used 


Group and PSBT Lesson number Assistive teaching Teaching approach PSBTs’ Knowledge level of 
and topic taught Technological tool(s) used selected TPACK domains 
1 -Wivine 1-Diffusion Phone and YouTube video Question and answer, TK-Basic 
Lecture method TPK-Basic 
TPACK-Basic 
2-Diffusion Computer, projector, YouTube Guided inquiry TK-Advanced 
video, PhET simulation TPK-Advanced 
TPACK-Basic 
3-Diffusion Computer, projector, YouTube Guided inquiry TK-Advanced 
Video, PhET simulations and TPK-Advanced 
Google form assessment TPACK -Advanced 
2 -Mugabo 1-Digestion Computer and YouTube video Teacher’s exposition TK -Advanced 
Question and answer TPK-Basic 
TPACK- Basic 
2-Digestion Computer, projector, PPTS, Whole class discussion TK -Advanced 
animations Question and answer TPK -Advanced 
TPACK -Basic 
3-Digestion Computer, PPTs projector, Whole class discussions TK - expert 
Animation and Google form and Teacher-led demon- TPK -Advanced 
assessment strations TPACK -Advanced 
3-Venuste 1-DNA replication Computer, projector, PPTS Teacher's exposition TK -Basic 
Question and answer TPK- Basic 


TPACK- Limited 


2-DNA replication 


Computer, projector, animation 


Question and answer 
Discussion 


TK -Advanced 
TPK- Basic 
TPACK- Basic 


3-DNA replication 
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Computer, projector, Virtual 
laboratory 


Demonstration, whole 
class discussion 


TK -Advanced 
TPK -Advanced 
TPACK -Advanced 
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Group and PSBT Lesson number Assistive teaching PSBTs’ Knowledge level of 


Teaching approach 


and topic taught Technological tool(s) used selected TPACK domains 
4 -Seraphine 1- Plant Cell structure Computer, projector, PPTS Teacher’s exposition TK - Basic 
TPK -limited 
TPACK -Limited 
2- Plant Cell structure © Computer, projector, PPTS Teacher’s exposition TK-Basic 
TPK-Basic 
TPACK-Basic 
3- Plant Cell structure | Computer, projector, PPTS Teacher’s exposition TK -Advanced 
TPK-Basic 
TPACK-Basic 
5-Andre 1-Mitosis Computer, projector, PPTs Teacher’s exposition TK- Basic 
TPK -Basic 
TPACK -Limited 
2-Mitosis Computer, projector, YouTube Teacher exposition, Q&A TK- Basic 
video embedded in PPTs TPK- Basic 
TPACK -Basic 
3-Mitosis Computer, projector, PRET Q &Aand teachers’ TK -Advanced 
simulation exposition TPK -Basic 
TPACK-Basic 


Results from PSBTs’ Micro-teaching Lesson Study. 


The changes in PSBTs’ TPACK through the different micro-teaching lesson study phases are depicted graphi- 
cally in Figure 3. 


Figure 3 
PSBT’s Performance in Micro-teaching Lesson Study 
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The results indicate that the lowest overall TPACK score for the PSBTs in the first lesson was 27 (Seraphine and 
Venuste) with 31 (Andre and Wivine) being the highest. For lesson two, the lowest recorded overall TPACK score 
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was 28 (Andre) while the highest was 37 (Wivine). Lesson 3 on the other hand recorded 30 (Seraphine) as the low- 
est overall TPACK score, while Mugabo (41) recorded the highest overall TPACK score. The average TPACK score 
for lesson 1 was 29, lesson 2 averaged 33.4 with lesson 3 averaging 36.4. It is clear that the PSBTs’ TPACK scores 
improved from lesson 1 to lesson 3. However, it is important to observe that some PSBTs’ TPACK scores did not 
increase systematically across the three lessons. Andre, for example, had an overall TPACK score of 31 in his first 
lesson but regressed to 25 in his second and then improved to 34 in his third lesson. Similarly, Seraphine’s TPACK 
score in the first lesson was 27 while the second lesson recorded her highest TPACK score of 32 with the third les- 
son regressing to an overall TPACK score of 30. 


PSBTs’ TPACK Evolution after Training 
In order to show the impact of the TPACK-ID model based technology integration course on PSBTs’ TPACK, 
PSBTs’ representative quotes before and after course implementation were picked to represent the evolution in 


the different domains of TPACK. The evolution of PSBTs’ TPACK after training is shown in Table 4. 


Table 4 
Evolution of PSBTs’ TPACK after Training 


Sample excerpts 


Themes Codes Sample interview questions 
Before training After training 

TK Using a projector, creating Mention some digital technological “Phone, Power “PowerPoint Presentation, You- 
PPTs, Creating and using tools (including software programs) Point presentation, Tube videos, PhET Simulations, 
YouTube Videos, manipulating — that you have used/can use for teach- YouTube videos’. Virtual laboratories, and many 
software programs like PhET _ ing and learning biology? more”. -Venuste 
simulations, PPTs confidently 

PK Knowledge of biology teaching Do you know various teaching “Discussion method, “Discussion, practical demon- 
approaches and assessment __ strategies that youcan use to teach _ practical method” strations, guided inquiry’-Wivine 
techniques biology? If yes, mention then 

TPK Selecting appropriate Do you know specific technologies “Magnification, we “Yes, for example | can use a 
technological tool for teaching _ that you can use for a particular are going touse the _ virtual lab to do a practical dem- 
strategy teaching strategy? If yes, give an technology which is _onstration on DNA replication’- 

example a microscope using Mugabo 
practical activity”. 

TCK Choosing appropriate technol- Do you think that in different topics “lam not really sure.” “Yes, different topics require 
ogy for a specific biology of biology, the use of technology different modes of presenta- 
concept materials would be different? Why? tion in order to make it easy for 

Why not? the understanding of pupils” 
-Seraphine 
TPACK Use of appropriate tech- Are you able to identify a specific “lam not really sure.” “Yes, maybe on DNA replication | 


nologies to represent biology 
concepts 


teaching strategy suited to a specific 
technological tool for teaching a 
specific biology topic? If yes, give an 
example. 


will use PowerPoint to show how 
DNA is replicated” -Venuste 


It can be seen from Table 4 that PSBTs’ TPACK had evolved in most TPACK domains. The PSBTs became aware 
of more technologies for teaching and learning, their PK also evolved as they became aware of different strategies, 
while their TPK, TCK and TPACK also showed improvements as they were able to make connections between the 
concepts to be taught, technological tool to be used with the appropriate teaching approach. 


Discussion 


The study examined PSBTs’ TPACK before and after a technology integration course based on the TPACK-ID 
model. Prior to the course implementation, PSBTs had a basic level of TK, TCK, TPK, and TPACK. The PSBTs demon- 
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strated a limited understanding of how to use technology effectively in teaching specific biology topics. However, 
after the course, their TPACK level significantly improved, with the majority of PSBTs demonstrating advanced 
level TPACK. 

Before course implementation, the study established that PSBTs’ had a basic understanding of technologies 
relevant to teaching and learning of biology, however, they did not realize how these technologies could be lever- 
aged for teaching and learning of biology. It is also worth noting that although PSBTs are active users of technol- 
ogy, they are not a homogeneous group: they possess different abilities in understanding and using technology. 
After course implementation, PSBTs’ technology knowledge showed a significant improvement in their TK as they 
became aware of technologies like PhET simulations and Google forms that they were unaware of before course 
implementation. This improvement can be attributed to support provided by experts during training on technolo- 
gies for teaching and learning, collaboration with peers and discussions on the proper use of technologies. The 
results of this study are consistent with Aktas and Ozmen (2020b, 2020a) who found that PSTs are not familiar with 
technologies such as simulations and that providing PSTs with training on new technologies for teaching helps 
to improve their TK and TPK. 

In terms of PK, it was observed that PSBTs used more teacher-centered teaching methods prior to course 
implementation. PSBTs predominantly used teacher exposition and lecture method as methods of instruction. 
This limited the effective use of technology in their teaching and consequently inhibited their TPACK develop- 
ment. Lee and Kim (2014a) averred that improving PSTs’ PK is key to developing their TPACK. The course training 
exposed PSBTs to technology-oriented approaches such as guided inquiry and teacher led demonstration. Also, 
by observing the instructor demonstrate a technology lesson, and having peer discussions on the best methods 
for lesson presentation, the PSBTs enhanced their PK which contributed to their improvement in choosing the 
appropriate strategy for using a particular technological tool (TPK). Furthermore, having opportunities to pres- 
ent a technology-rich lesson enhanced PSBTs competencies in teaching with technology. The findings support 
studies by other researchers who reported that having an instructor role model and providing technology-based 
education to PSTs has the potential to improve their TPK and pedagogical skills (Bwalya et al., 2024; Irmak & Yilmaz 
Tuztin, 2019; Lee & Kim, 2014a, 2014b). 

In terms of TCK, it was observed that prior to course implementation, PBSTs did not consider the topic they 
were teaching when selecting the technology to use for teaching. The PSBTs chose the technology primarily based 
on which technology they knew and not the topic they were supposed to teach. This led to most PSBTs using PPTs 
and YouTube videos as those were the only instructional technologies, they were familiar with. Nevertheless, after 
course implementation, PSBTs realized the need for suiting content being taught with the appropriate technology. 

The findings indicate that the TPACK-ID model-based technology integration course improved PSBTs’ TPACK 
competencies by enhancing their ability to select, match, and effectively use technology in teaching biology. Having 
PSBTs to define and discuss TPACK, collaboratively design lessons, and implement, reflect and revise technology- 
rich lessons were the major factors which were noted to contribute to PSBTs’ TPACK development. The findings of 
this study are consistent with previous studies (e.g. Jimoyiannis, 2010; Lee & Kim, 2014a, 2017; Srisawasdi, 2014; 
Srisawasdi et al., 2018) which found that implementing well-designed coursework could foster PSTs improvement 
in their TRACK. 


Conclusions 


In conclusion, the TPACK-ID model-based technology integration course effectively enhanced the TPACK 
of PSBTs. The study findings indicate that PSBTs improved in their TK, PK, TPK, TCK and overall TPACK. The PSBTs 
demonstrated a combined form of PK, CK and TK (Integrative TPAC) rather than the homogenous form of TPACK 
(Transformative TPACK). The course activities such as training on TPACK and technologies for teaching and learning 
biology, providing concrete examples on the use of technology for biology teaching, PSBTs’ collaborative lesson 
design and implementation, reflecting on lessons, and iterative revisions of lessons helped in enhancing the TPACK 
of PSBTs. It should be noted that the development of TPACK in PSTs is not a one-off process, instead, PSTs must 
continue to practice teaching with different technologies to keep improving their expertise of teaching effectively 
with technology. Additionally, it is worth noting that technology and technology use to support teaching and 
learning is in a constant state of flux, that is, it is always changing hence the need to continuously learn about new 
technologies and how they can be utilized to enhance teaching and learning. 
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Limitations 


The main limitation of this study is that its findings are from one university and in a biology context thus, do not 
allow for generalization. Nevertheless, the study lays a good foundation for the strategies useful for developing PSTs’ 
TPACK in specific subject contexts. The other limitation is that the study used qualitative findings without triangulating 
them with quantitative results. However, the research is strengthened by triangulating findings from different data 
sources such as interviews, lesson plan reports, video lesson recordings and reflection notes to improve its reliability. 


Recommendations 


The study found that the PSTs’ teachers were not trained on technology integration in teaching because there 
was no course in their curriculum designed to meet this need. There is, therefore, a need for the TTIs to design and 
integrate TPACK-based courses for technology integration into teaching in their curriculum. The proposed TPACK- 
ID model-based technology integration course can be implemented in different contexts and at different TTIs to 
enhance PSTs’ competencies in teaching with technology. Further, there is a need to carry out longitudinal studies 
to get more in-depth information on how PSTs develop their TPACK. 
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Appendix 1 
Sample Questions from the Pre and Post-test Interview Guide 


Sample questions 
3. Mention some digital technological tools (including software programs) that you have used/can use for teaching and 
learning biology. 
. How can you self-evaluate your knowledge of digital technologies used in biology classroom instruction? 
. Are there any topics in biology that you struggle to understand and teach? If yes, mention them. 


. Do you know various teaching strategies that you can use to teach biology? If yes, mention then 


. Are you able to identify specific strategies for teaching a particular concept in biology? 


4 
5 
6 
7. Do you know specific technologies that you can use for a particular teaching strategy? If yes, give an example. 
8 
9. While teaching biology, do you think that technology is necessary? If yes, explain more. 

1 


0. Are you able to identify a specific teaching strategy suited to a specific technological tool for teaching a specific 
biology topic? If yes, give an example. 


11. Do you think the TPACK course you participated in helped you to develop useful skills/competencies for your 
biology teaching? Yes, or NO? If yes, explain. (Only asked in the post-intervention interview). 


12. If your answer to question 9 above is yes, which component/s of the TPACK course (Training on TPACK frame- 
work and technologies for teaching and learning biology, Instructor lesson demonstration, collaborative lesson planning, 
lesson presentation, Reflection on lesson and revisions) helped you to gain the skills/competencies you have mentioned? 


(PP.956-973) 


TPACK domains 


TK 


TK 
CK 
PK 
TPK 
PCK 
TCK 
TPACK 


Impact of TPACK-ID 
model course 


Impact of TPACK-ID 
learning strategies 
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Abstract. Various studies have reported 
that students have limited, stereotypical 
perceptions of STEM fields. However, 

few studies have attempted to evaluate 
interventions undertaken with the aim of 
changing female student's perceptions of 
STEM professions. This research aimed to 
examine the changes in mental images 
among female students as a result of 
interacting with STEM professionals ina 
STEM camp. The Draw-a-Scientist Test was 
revised, and female students were asked 

to draw and describe STEM professionals 
at work before and after attending STEM 
camp. The participating students attended 
workshops and career talks given by STEM 
professionals. The findings showed that 
the perceptions of the participating female 
students regarding gender images of STEM 
professionals changed. After the STEM 
camp, it was also observed that knowledge 
about STEM professionals and the content 
of their work had increased. The studied 
intervention program for female students 
positively contributed to changes in 

the students’ mental images of STEM 
professionals. Therefore, it is recommended 
that female students interact with 
professionals working in STEM fields to 
change their perceptions of those fields. 


Keywords: female students, gender 
stereotypes, human images in STEM, role 
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Introduction 


As students gain experience as learners, many of them imagine their 
possible selves in terms of what they do and do not want to be in the future 
(Markus & Nurius, 1986). In spite of growing demands for a better under- 
standing of science, technology, engineering, and mathematics (STEM) 
within communities (National Research Council [NRC], 2011) and more 
STEM-literate citizens (Zollman, 2012; Huang et al, 2022), there seems to 
be reluctance among students to participate in STEM fields (Stephenson et 
al., 2021). A large body of research has addressed ongoing concerns about 
gendered professional participation and the underrepresentation of women 
in STEM-related occupations, focusing on international statistics, the depar- 
ture of women from STEM fields, and disinterest among female students in 
pursuing STEM careers (United Nations Educational, Scientific and Cultural 
Organization [UNESCO], 2020; National Science Foundation [NSF], 2019). The 
underrepresentation of women in STEM fields is attributed in part to early 
gender socialization, such as the perpetuation of stereotypes, during a de- 
velopmental stage in which academic identities and interests are emerging 
(Rogers et al., 2021). It is important to understand how the disintegration of 
the STEM identities of female students begins in order to stop the process 
and guide more female students toward successful STEM careers (Corrigan & 
Aikens, 2020). The forms of interaction of people in society both reflect and 
influence the formation of their identities, beliefs, values, and choices, which 
are affected by stereotypes that are transmitted explicitly and implicitly from 
childhood. This leads girls to believe that STEM subjects in general are for boys, 
which in turn affects girls’ confidence and interest in studying STEM-related 
topics. Despite the significant concern surrounding the underrepresentation 
of female students in STEM classes at both grade schools and universities, 
it’s worth noting that prior research on gender differences among students 
in classrooms that implement integrated STEM projects remains relatively 
limited compared to the broader body of research on gender differences in 
science education (Koul et al., 2021). 
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Within this context, informal STEM education programs have become spaces where action and advocacy 
can take place in safe settings while students work with role models who resemble them and participate in STEM 
projects with professionals from these fields. Studies have shown that interventions such as science camps shape 
students’ images of scientists (Leblebicioglu et al., 2020). Fostering a social environment where female students can 
identify with successful female role models in STEM can help change stereotypes about women in STEM. The aim 
of the present study is to examine the effects of a career-based STEM teaching model on female students’ percep- 
tions of STEM professions. After providing the theoretical background, the research findings from an experimental 
STEM education intervention program for female upper-secondary students are discussed to evaluate whether this 
model creates conditions for female students to develop positive images of STEM professions. 


Literature Review 
Gender Gap in STEM 


As women constitute an average of 30% of research and development employees and only 18% of relevant 
task forces, they are underrepresented in these fields in many countries around the world (UNESCO, 2020). In the 
United States, 18.7% of all students earning a bachelor’s degree in computer science are women, while this rate is 
20.9% in engineering, 42.4% in mathematics and statistics, and 19.3% in physics (NSF, 2019). Female students often 
begin to question their sense of belonging in STEM fields, also known as STEM identity, during adolescence (Roberts 
& Hughes, 2019). Accordingly, there are significant decreases in the percentage of female students who choose to 
pursue STEM subjects or show interest in STEM professions, especially after starting upper-secondary (Chen et al., 
2022). It has been shown that gender differences in STEM career interests are common among upper-secondary 
students (Mau et al., 2020). Hence, it is imperative to consistently explore strategies for nurturing and sustaining 
STEM aspirations among students, irrespective of their gender, throughout their educational journeys (Chen et al., 
2022). The notion that STEM fields entail limited interpersonal interaction and the perpetuation of gender-biased 
stereotypes about STEM professionals in mainstream media can potentially hinder the development of a strong 
STEM identity, particularly among adolescent girls (Master & Meltzoff, 2020). Pilotti (2021) offered a nuanced 
perspective on the various forms of gender bias and recommended the implementation of tailored educational 
interventions aimed at enhancing the motivation and enrollment of women in STEM disciplines. 


Images of STEM Professions 


As the significance of scientific knowledge continues to gain prominence, nations are increasingly focusing 
their efforts on augmenting both the quantity and caliber of scientists who play a pivotal role in advancing our 
understanding of the world through scientific research (Karacam et al., 2021). The global competitiveness of coun- 
tries is often suggested as a key outcome of developing a skilled STEM workforce (Bybee, 2013; Constan & Spicer 
2015). Studies have shown that students have stereotypical images of scientists; for example, scientists are often 
portrayed as men wearing lab coats, with curly hair, beards, and glasses, working with test tubes in a laboratory 
(Ates et al., 2020). It is not easy to change that stereotypical image of scientists because individuals, and especially 
women, are exposed to such stereotypes for years through the media, textbooks, and social media (Karacam et al., 
2021). A close connection is perceived to exist between science and masculinity in our society today, historically 
informed by masculine conceptions of scientific professions (Archer et al., 2013). In the literature, there is growing 
interest in students’ perceptions of scientists as a result of the recognition that stereotypical perceptions of scien- 
tists may influence students’ attitudes toward science and their interest in science-related activities and careers 
(Emvalotis & Koutsianou, 2018). Finson (2002) emphasized that an individual’s mental depiction of scientists holds 
significant sway over their inclination to pursue a career in scientific fields. This is because there exists a profound 
interconnection between one's mental imagery of scientists and their attitudes towards science, as well as their 
self-assurance in engaging in scientific pursuits. 

Generally speaking, sufficient research has not yet been conducted on professions in the field of technology 
(Silver & Rushton, 2008). It has been stated that students tend to have positive attitudes toward technology coupled 
with a limited understanding of the field (Ankiewicz, 2019; Koch, 2013). For example, in a previous study conducted 
with secondary school students, a representative image of a technologist depicted a man with short hair wearing 
safety glasses and a welding mask (Jung & Kim, 2014). Technologists were described as working with cars, robots, 
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and rockets and using wrenches, hammers, screwdrivers, and welding machines (Jung & Kim, 2014). It has also 
been reported that students confuse people who work with technology and engineers. To develop a rational and 
comprehensive perception of technologists, students must receive effective STEM education and acquire a deep 
understanding of the nature of each STEM component (Blom & Abrie, 2021). 

Various studies have revealed that children tend to portray engineers as white, male, and self-employed people 
who do physical work on a large scale, such as repairing and building things (Chou & Chen, 2017). In a report issued 
by the National Academy of Engineering (NAE) (2008), it was observed that while many students held a generally 
favorable view of engineers, they expressed self-doubt about their own abilities, believing that they might not be 
intelligent enough to pursue a career in engineering. Furthermore, existing literature frequently highlights that 
students possess a restricted comprehension of engineering and the tasks performed by engineers (Kelly et al., 
2017; Knight & Cunningham, 2004). Many students have misconceptions about engineers (Fralick et al., 2009), 
and recent studies indicate that even preservice teachers have stereotypical images of this profession (Kuvac & 
Koc, 2022). Hence, it is of utmost importance to dispel misconceptions surrounding STEM careers through the 
dissemination of precise information (Wyss et al., 2012) and the integration of additional STEM resources into the 
educational curriculum (DeJarnette, 2012). A significant number of K-12 students tend to associate engineers with 
occupations such as train drivers, auto mechanics, construction workers, or individuals operating large machinery 
(Capobianco et al., 2011). When students’ drawings included identifiable gender representations, the figures were 
predominantly male. Furthermore, a higher percentage of male figures were present in students’ drawings depict- 
ing engineers compared to those representing scientists (Fralick et al., 2009; Karatas et al., 2011). 

Math images are important elements for students to solidify their knowledge about math and their math 
learning. Students’ understanding of such concepts is molded by a range of activities and diverse life experiences, 
including those related to mathematics education (Osman et al., 2010). Rock and Shaw (2000) asserted that when 
students perceive mathematics as an unappealing subject through their perceptions of mathematics and math- 
ematicians, the process of mathematics education becomes notably more arduous. Picker and Berry (2000) found 
that students’ images of mathematicians reflected balding and messy men, and it is widely recognized that men are 
represented more than women in the fields of mathematics and science (Gjavik et al., 2022). If the representations 
of STEM fields that children and young adults encounter negatively affect their STEM-related self-concepts and 
identity development, the effects may spill over to negatively influence their continued involvement and partici- 
pation in these fields. Therefore, the portrayal of gender role stereotypes of scientists, technologists, engineers, 
and mathematicians in media representations may play a significant role in shaping gendered identities (Steinke 
et al., 2007). From this point of view, it is important for students to encounter professionals of both genders while 
forming their perceptions of STEM fields. Consequently, the favorable perception of STEM professionals and the 
understanding of STEM’s implications for society among students are significantly correlated with their interest in 
pursuing STEM careers (Pan et al., 2022) 


Role Models in STEM Education 


Efforts aimed at increasing the involvement of historically marginalized groups in STEM frequently emphasize 
the importance of showcasing real-life STEM role models. This approach serves to confront and dismantle stereo- 
types while fostering a sense of connection with these fields (Van Camp et al., 2019). In this context, a role model 
is defined as someone who is more advanced in technology or engineering and has long-term interactions with 
a less advanced person who is making a decision (Guenaga et al., 2022). Effective informal STEM education initia- 
tives create secure physical and psychological settings for girls, enabling the cultivation of positive social norms, 
encouraging supportive peer and role model interactions, and fostering a sense of belonging (Simpkins et al., 
2017). Research conducted with adolescent girls demonstrated that their interest in STEM disciplines increased, 
and their self-perceptions regarding science and mathematics improved following their participation in a summer 
camp featuring authentic female role models (Hughes et al., 2013). Counteracting the ease with which stereotypes 
are transmitted and the ways in which they affect the performance of female students in STEM fields, female role 
models can serve as constant reminders that STEM fields are for everyone and may help motivate young women 
to pursue careers in STEM-related fields (Fernandez-de la Pena et al., 2021). The advantages of such programs are 
remarkable, as scientists can serve as positive role models, and engagements with even brief outreach initiatives 
can yield enduring positive effects on students. This includes an increased interest in pursuing careers in the field 
of science (Kompella et al., 2020). For many students, gaining access to real-life STEM role models representing 
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diverse backgrounds can be challenging in STEM learning environments (Steinke et al., 2022). Therefore, provid- 
ing access to role models and opportunities to work with professionals in these fields can create opportunities to 
understand and change students’ perceptions of STEM careers. Moreover, effective initiatives facilitate the devel- 
opment of STEM identities by integrating inclusive curricula and teaching methods with opportunities to interact 
with female role models (Prieto-Rodriguez et al., 2020). 


Out-of-School Learning Environments 


Teaching and learning about scientific research helps students better understand the nature of science and 
the work of scientists (Emvalotis & Koutsianou, 2018). Therefore, more systematic education and training interven- 
tions are needed to facilitate more accurate and realistic images of the work of scientists (Emvalotis & Koutsianou, 
2018). Young people today can take advantage of a wide variety of opportunities to engage with STEM content 
and educational activities that will be unquestionably beneficial to them as future citizens (Zollman, 2012). It was 
also shown that out-of-school projects may have the potential to increase interest in science among young people 
and that female students are particularly interested in artistic and realistic educational activities (Walan, 2021). The 
number of girls and women pursuing STEM-related careers is slowly increasing thanks to extracurricular activities 
that take new approaches to introducing STEM to children of all ages (Fernandez-de la Pena et al., 2021). Sadler et 
al. (2012) proposed that initiatives aimed at nurturing female students’ enthusiasm for mathematics and science 
have the potential to concurrently enhance their interest in pursuing careers in STEM fields upon completing 
upper-secondary education. Exposing girls to various female role models in out-of-school learning environments 
expands the representation of women in these areas and allows students to imagine themselves in the place of 
those role models (Anderson & Cavallaro, 2002). A study conducted by Gonzalez-Pérez et al. (2020) assessed the 
impact of a role-model intervention involving female volunteers working in STEM who visited schools to discuss 
their careers with female students. The results of the intervention were notably positive, with a significant increase 
in the girls’ confidence in their ability to succeed in STEM fields, as well as an elevated likelihood of choosing a STEM 
career (Gonzalez-Pérez et al., 2020). Such programs provide students with opportunities to engage in authentic 
real-world problem-solving activities that go beyond the content covered in textbooks (Baran et al., 2019). They 
expose students to STEM concepts and ideas in informal learning environments that may not be readily available 
within the formal school curriculum (Baran et al., 2019). 


Theoretical Background 


Social Cognitive Theory (SCT) is a psychological framework that explains human behavior and learning 
processes through the interaction of personal, behavioral and environmental factors (Bandura, 1986). This theory 
suggests that individuals acquire knowledge and skills by observing others, drawing on personal experiences and 
responding to environmental influences (Bandura, 1997). In the context of STEM education, SCT can be utilized to 
understand and improve students’ engagement, motivation, and performance in these fields, particularly among 
female students who are underrepresented in the professions (Cheryan et al., 2017). SCT emphasizes the importance 
of role models and mentors in shaping students’ attitudes, beliefs, and behaviors (Bandura, 1986). In the context of 
STEM education, exposure to diverse and successful STEM professionals, especially female role models, can help 
students develop positive perceptions of these fields, challenge stereotypes, and see STEM careers as viable op- 
tions. Informal STEM education programs and out-of-school learning environments such as science camps provide 
students with opportunities to interact with and learn from STEM role models, providing a supportive learning 
environment that encourages students, especially girls, to pursue STEM subjects and careers (Dabney et al., 2012). 
According to SCT, the learning environment plays a crucial role in shaping students’ experiences and outcomes 
(Bandura, 2001). In the context of STEM education, stereotypes about scientists, technologists, engineers, and math- 
ematicians in media portrayals can negatively impact the formation of gendered identities and contribute to the 
underrepresentation of women in STEM fields (Cheryan et al., 2017). A supportive and inclusive environment that 
values diversity, collaboration, and innovation can promote student engagement and achievement (Simpkins et 
al., 2017). Socio-cognitive career theory examines the process by which students imagine their future occupations 
and the social influences on this process. However, this theory has generally not addressed important factors such 
as gender roles and gender norms. This research examines in more detail the gender-related factors that influence 
students’ interest and engagement in STEM fields. 
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Research Aim and Research Questions 


Adolescence is commonly recognized as a crucial developmental stage for challenging and dispelling cultural 
stereotypes (Cvencek et al., 2015). It is also a crucial period in terms of students’ understanding of STEM fields and 
their decisions about pursuing STEM careers (Lampley et al., 2022). Increasing the STEM experiences of adolescents 
and planning activities in partnership with professionals working in STEM fields are supportive elements in orient- 
ing students toward STEM fields. Stereotypical views of science and scientists can detrimentally affect the career 
prospects and opportunities of upper-secondary school students (Garriott et al., 2017). Conversely, engaging stu- 
dents in discussions about a range of STEM career possibilities can aid in the formation of their future aspirations 
(Constan & Spicer, 2015). A lack of positive role models may prevent students from achieving their career goals 
(Constan & Spicer, 2015), while the presence of role models and out-of-school learning programs may provide 
support for upper-secondary students’ STEM career goals, positive perceptions of scientists, and general interest 
in the sciences (Barry et al., 2022). The present study was undertaken with the aim of overcoming gender stereo- 
types of STEM professionals held by female upper-secondary students by emphasizing diversity in STEM fields in 
an interdisciplinary way. Female upper-secondary students’ perceptions of people working in STEM fields and the 
changes in those perceptions before and after attending a STEM camp were explored within the framework of 
the following research questions: 

RQ1: How do female students’ perceptions of scientists differ before and after STEM camp? 

RQ2: How do female students’ perceptions of technologists differ before and after STEM camp? 
RQ3: How do female students’ perceptions of engineers differ before and after STEM camp? 

RQ4: How do female students’ perceptions of mathematicians differ before and after STEM camp? 


Research Methodology 


This research was designed as a case study, an approach selected from among qualitative research methods. 
Case studies are used to understand people's lives and experiences, and they involve an in-depth and detailed 
examination of particular situations in the context of real phenomena or events (Bromley, 1986). In a phenomeno- 
logical case study, the researcher seeks to gain insight into the subjective perspectives, thoughts, emotions, and 
interpretations of the participants involved in the case. The goal is to capture the essence of their experiences and 
provide a rich and detailed description of their lived reality (Han, 2015). The phenomenological case study allows 
participants to describe their experiences of attending the STEM camp and the impact of these experiences on 
their perceptions. 


Research Process and Participants 


The research was conducted with 25 female upper-secondary students over the course of 6 days. Five stu- 
dents (20%) were 16 years old, and 12 students (80%) were 17 years old. The research was funded by the TUBITAK 
(The Scientific and Technological Research Council of Turkiye), and ethical approval was obtained from the ethics 
committee of the researcher's university. Permission to conduct the research was also obtained from the local 
authorities of the Ministry of National Education (MoNE). Students voluntarily offered to participate in the project 
by filling out an application form online. Subsequently, 113 possible participants were selected by the researcher 
from among those applications. The applicants were then selected based on specific criteria. These criteria included 
factors such as age and previous STEM experiences. In this way, the results were intended to become more specific 
and meaningful, given that the group of participants represented in the research had a specific demographic. After 
the students volunteered to participate in the project, their families were also contacted for permission (N= 25). 
This research was conducted at a university in Turkey in September 2021. 
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Table 1 
STEM Camp Schedule 
First Day Second Day Third Day Fourth Day Fifth Day Sixth Day 
Pre-Test Drama Robotic Coding Traditional Arts STEM Activity: Music in STEM 
“Draw a STEMist” (Male) with Lego and with STEM Are You Up For Education 
mBot (Male) (Male) The Marshmallow (Female) 
Duel? 
Morning Ses- (Female) 
sions 
Drama Socio-Scientific Physics Applica- Traditional Arts Robotic Coding Argumentation in 
(Male) Issues and STEM __ tions with Arduino with STEM with Lego and STEM 
Practices (Female) (Male) mBot (Male) (Male) 
(Male) 
Nature in Science Science Activity: Science Activity: Traditional Arts Socio-Scientific Physics Applica- 
and STEM Education DNAand Genetic © DNAand Genetic with STEM Issues and STEM _ tions with Arduino 
(Male) Code Design Code Design (Male) Practices (Female) 
(Male) (Male) (Male) 
Afternoon Ses- Career Talks: Career Talks: Career Talks: Career Talks: Career Talks: Post-Test: 
sions Nanotechnology Computer Expectations of Traditional Arts Health Practices “Draw a STEMist” 
(Female) Engineering and Business Life in (Female) (Female) 
Artificial Intelligence the Future 
Applications (Female) 
(Female) 


Table 1 shows the educational content of the STEM camp and the genders of the presenters. Seven male and 
four women took part in the educational training sessions. Each day of the camp focuses on different STEM topics 
and activities, complemented by career talks and pre-and post-tests to measure students’ perceptions of STEM 
professionals. The camp provides students with the opportunity to experience STEM fields and also allows them 
to interact with STEM professionals. 


Data Collection Tools 


Children’s drawings have long been an object of study for researchers in many fields (Longobardi et al., 2017). 
In this study, the Draw-a-Scientist Test (DAST-C) was used to determine students’ perceptions of people working 
in STEM fields, and three alternative tests (Draw a technologist, draw an engineer, draw a mathematician) were 
used to describe professionals in other fields. The scientific environment portrayed, and the gender and physical 
characteristics of the represented scientists were used to evaluate the results of the classical DAST-C, and similar 
features were used in evaluating the three alternative tests. Thus, four separate forms were given to the students, 
and they were respectively asked to draw a scientist, a person working in the field of technology, a person working 
in the field of engineering, and a person working in the field of mathematics. Instructions on each form asked them 
to explain the drawing and the characteristics of the person they drew as it would have otherwise been difficult 
to evaluate their drawings. During the drawing process, students were provided colored pencils and paper, and 
one hour was allotted for the creation of the drawings at the beginning and the end of the STEM camp. Instructors 
were present in the classroom but did not provide any additional time or instructions to the students. 


Data Analysis 


The data analysis included quantification, which refers to the conversion of qualitative data into numerical 
values (Sandelowski et al., 2010). Quantification of qualitative data is a good tool in thinking about and interact- 
ing with data. A checklist was created for evaluating students’ drawings in each of the STEM disciplines, drawing 
upon tools and methodologies outlined in existing literature (Meyer et al., 2019; Martins et al., 2021). For example, 
Martins et al. (2021) applied a checklist developed by students to evaluate their drawings of professionals work- 
ing in STEM fields. Chambers (1983) and Finson et al. (1995) also evaluated participants’ drawings using a scoring 
checklist of elements that serve as indicators in standard images of scientists. In the present study, the drawings 
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and the participants’ explanations of those drawings were analyzed by two science education experts according 
to the analytical framework proposed by Fralick et al. (2009). These experts evaluated the drawings according to 
a specific analytical framework and took note of their characteristics, such as content, gender representation and 
physical characteristics. A reliability formula (Reliability = Consensus / (Consensus + Disagreement)) was applied 
to their results and the agreement rate between the two experts was determined as 89.9% (Miles & Huberman, 
1994). A descriptive analysis of the results was performed based on the recorded frequencies for each item of the 
checklist for students’ drawings made before and after the STEM camp. According to the results of the Shapiro-Wilk 
normality test (p < .05), non-parametric tests were used because the data were not normally distributed. Percent- 
ages, frequencies, chi-square and Wilcoxon tests were applied to compare the pre-test and post-test drawings. In 
order to evaluate the items in the checklist, a scoring system was applied by giving a value of 1 (yes) or 0 (absent) 
and calculating (Chambers, 1983; Finson et al. 1995; Martins et al. 2021). The classification criteria created by Martins 
et al. (2021) were used to evaluate the dimensions of technology, engineering, and mathematics. 


Research Results 
Perceptions of Scientists Before and After STEM Camp 


This sub-section presents an analysis of female students’ perceptions of scientists before and after attending 
the STEM camp. The research question is defined as RQ1, and this section focuses on the participants’ perceptions. 


Table 2 
Students’ Drawings of Scientists 


Dimensions Items wwo2s We 2) pene 
f (%) f (%) x p 
1. Lab coat 14 (56) 4 (16) 5.82 0.001* 
2. Eyeglasses 10 (40) 5 (20) 2.38 0.12 
3. Facial hair 6 (24) 2 (8) 2.38 0.12 
4. Symbols of research 10 (40) 11 (44) 0.82 0.77 
5. Symbols of knowledge 8 (32) 14 (56) 0.76 0.38 
6. Technology 10 (40) 11 (44) 0.82 0.77 
7. Relevant captions 1 (4) 3 (12) 1.08 0.29 
DAST-C Items 8. Male gender 12 (48) 3 (12) 771 0.001* 
9. Indications of danger 4 (16) 2 (8) 0.75 0.38 
10. Presence of light bulbs 4 (16) 2 (8) 0.75 0.38 
11. Mythical stereotypes - - 0.00 1.00 
12. Scientists doing work indoors 16 (64) 7 (4) 6.52 0.01* 
13. Middle-aged or elderly scientist 14 (56) 4 (16) 8.61 0.001* 
14. Crazy hair 6 (24) 5 (20) 0.11 0.73 
15. Bald head 3 (12) 1 (4) 1.08 0.29 
16. Male 12 (48) 3 (12) 7.71 0.001* 
eau 17. Female 1 (4) 12 (48) 12.58 0.001* 
18. Undetermined 12 (48) 7 (28) 2.12 0.14 
19. No person - 3 (12) 3.19 0.07 
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Dimensions Items wwo3s We 2) maciaias 
f (%) f (%) x p 
Expression 20. Smiling 1 (4) 5 (20) 3.03 0.08 
21. Frowning 8 (32) 2 (8) 4.50 0.03* 
22. Physics 12 (48) 16 (64) 1.29 0.25 
Field of specialization 23. Chemistry 6 (24) 15 (60) 6.65 0.10 
24. Biology 7 (28) 12 (48) 1.38 0.23 
25. Omniscient - 2 (8) 2.08 0.14 
26. Indoors 20 (80) 8 (32) 11.68 0.001* 
Woriolee 27. Outdoors 3 (12) 41 (44) 6.34 0.01" 
28. Undefined 2 (8) 6 (24) 2.38 0.12 
29. Making something/working with hands 8 (32) 18 (72) 8.01 0.001* 
30. Discovering/researching 6 (24) 17 (68) 9.74 0.001* 
31. Experimenting/testing 10 (40) 17 (68) 3.94 0.04* 
pein 32. Explaining/teaching 8 (32) 14 (56) 2.92 0.08 
33. Observing 6 (24) 10 (40) 1.47 0.22 
34. No action 1 (4) 2 (8) 0.35 0.55 
35. Other - - 0.00 1.00 


Table 2 presents the frequencies and corresponding percentages of the indicators that were examined during 
the analysis. The participants included almost all of the items from the DAST-C in the pre-test (M = 8.42; SD = 4.60) 
and post-test (M = 5.28; SD = 4.00). This indicates that they had stereotypical images of scientists before attending 
the STEM camp. Regarding the differences between the pre-test and post-test, the Wilcoxon test revealed statisti- 
cally significant differences (z = -2.080, p = .03). Among the items considered, there were changes in the images of 
lab coats, gender, scientists working inside, and middle-aged or elderly people. While 48% of the participants drew 
a male image in the pre-test, 12% drew a male image in the post-test and this difference was significant (y? = 7.71, 
p<.001). In the pre-test, a female figure was present in only 4% of the drawings, while in the post-test, 48% of the 
drawings featured female figures, again constituting a significant difference (y?= 12.58, p < .001). Participants also 
had changes in the emotions that they represented in their drawings. The drawings from the post-test featured 
fewer frowning images (yx? = 4.50, p = .03). No significant differences were determined for the represented fields for 
science, but a numerical increase was observed in the portrayals of the fields of physics, chemistry, and biology in 
the post-test. There was a decrease of percentages in portrayals of indoor activities from 80% to 32% (y? = 11.68, 
p<.001) and an increase in portrayals of outdoor activities from 12% to 44% (x? = 6.37, p < .001). There were also 
significant positive increases in portrayals of making things and working with one's hands (x? = 8.01, p < .001), 
discovering/researching (x? = 9.74, p < .001), and experimenting/testing (x? = 3.94, p = .04). 


Perceptions of Technologists Before and After STEM Camp 


This sub-section presents an analysis of female students’ perceptions of technologists before and after attending 
the STEM camp. The research question is defined as RQ2, and this section focuses on the participants’ perceptions. 
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Students’ Drawings of Technologists 


Dimensions Items n= w= 25 ceases 
f (%) f (%) ¥ p 
4. Male 11 (44) 4 (16) 4.66 03" 
dei 2. Female 5 8 (32) 9.52 001* 
3. Undetermined 9 (36) 10 (40) 0.08 7 
4. No person 5 (20) 3 (12) 0.59 A4 
5. Smiling 6 (24) 7 (28) 0.10 74 
Paression 6. Frowning 4 (16) 2 (8) 0.75 38 
7. Facing backwards 8 (32) 2 (8) 4.50 .03* 
8. Computeritablet 20 (80) 20 (80) 0.11 73 
9. Cell phone 48 (72) 14 (56) 4,389 23 
Technology Symbols 19 Robot/drone : 21 (84) 36.20 001* 
11. None - - 
42. Other 6 (24) 4 (16) 0.50 48 
43. Indoors 14 (56) 3 (12) 10.78 001* 
Workolete 44. Outdoors 4 (24) 11 (44) 4.66 31 
45. Undefined 7 (28) 11 (44) 1.39 23 
46. Making/fixing/working with hands 42 (48) 14 (56) 0.32 57 
17. Discovering/researching 10 (40) 10 (40) 0.00 1.00 
easel 48. Explaining/Teaching 6 (24) 9 (36) 0.85 35 
49. No action 2 (8) 2 (8) 0.00 1.00 
20. Other 2 (8) 4 (8) 0.35 55 


In Table 3, findings from the students’ drawings of technologists are presented in terms of the dimensions, 
various features, and percentages. Regarding the variable of gender, it is seen that 44% of the participants drew 
male figures in the pre-test, while only 16% drew male technologists in the post-test (x? = 4.66, p = .03). While no 
women were drawn in the pre-test, 32% of the participants drew female figures in the post-test (x? = 9.52, p < .001). 
When portrayals of emotions were examined, no significant differences were observed. Turning to technology 
symbols, there was no difference in portrayals of computers, tablets, or phones between the pre-test and post-test, 
but portrayals of robots and drones were absent in the pre-test and were drawn by 84% of the participants in the 
post-test (x? = 36.20, p < .001). While 56% of the drawings from the pre-test featured people working indoors, 12% 
of the post-test drawings portrayed indoor locations (x? = 10.78, p < .001). On the other hand, 24% of the pre-test 
drawings and 44% of the post-test drawings portrayed people working outdoors. In the dimension of actions, 
increases were also observed in portrayals of people making/fixing/working with hands and explaining/teaching. 


Perceptions of Engineers Before and After STEM Camp 


This sub-section presents an analysis of female students’ perceptions of engineers before and after attending 
the STEM camp. The research question is defined as RQ3, and this section focuses on the participants’ perceptions. 
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Table 4 


Students’ Drawings of Engineers 


fines Wes Chi-squared Test 
Dimensions Items f (%) f (%) 9 p 
1. Male 8 (32) 5 (20) 0.93 52 
Bei 2. Female 2 (8) 8 (32) 4.50 03" 
3. Undetermined 15 (60) 7 (28) 5.19 02" 
4. No person - 5 (20) 5.55 .02* 
5. Smiling 5 (20) 5 (20) 0.00 1,00 
Sen 6. Frowning 1 (4) 4 (16) 2.00 15 
7. Facing backwards 2 (8) 2 (8) 0.00 1.00 
8. Building/fixing 8 (32) 15 (60) 3.94 05 
9. Designing 8 (32) 14 (56) 2.92 08 
Engineering Themes 10. Mechanical engineering 2 (8) 3 (12) 0.22 63 
11. Civil engineering 2 (8) 10 (40) 7.01 001" 
12. None 2 (8) : 2.08 15 
43. Other 2 (8) 8 (32) 4.50 03 
14. Indoors 40 (40) 5 (20) 2.38 12 
Moats 15. Outdoors 2 (8) 16 (64) 17.01 001" 
16. Undefined 13 (52) 2 (8) 11.52 00 
17. Building 40 (40) 17 (68) 3.94 05 
18. Fixing 6 (24) 13 (52) 4.16 04" 
pen 49. Projects/planning 4 (16) 12 (48) 5.88 02" 
20. No action 2 (8) 2 (8) 0.00 1.00 
21. Other 2 (8) 2 (8) 0.00 1.00 


Table 4 provides information about the students’ drawings of engineers before and after the STEM camp. In 
the pre-test, the gender of the figures was unclear in a majority of the drawings (60%). Male figures were depicted 
in 32% of the pre-test drawings and 20% of the post-test drawings. The portrayal of female figures increased from 
8% of all pre-test drawings to 32% of all post-test drawings (x? = 4.50, p= .03). Thus, there was a significant decrease 
in the participants’ portrayals of engineers of undetermined gender (x? = 5.19, p = .02). While no significant differ- 
ences were observed in portrayals of expressions, a significant increase was observed in portrayals of the field of 
civil engineering (x?= 7.01, p< .001). Regarding portrayals of workplaces, 40% of pre-test drawings involved indoor 
settings compared to 20% of post-test drawings. Depictions of work conducted outdoors increased significantly 
from 8% to 64% (x? = 17.01, p < .001). Finally, considering engineering practices, significant increases were seen 
in portrayals of engineers fixing things (x? = 4.16, p = .04) and working on projects/planning (x? = 5.88, p = .02). 


Perceptions of Mathematicians Before and After STEM Camp 
This sub-section presents an analysis of female students’ perceptions of Mathematicians before and after 


attending the STEM camp. The research question is defined as RO4, and this section focuses on the participants’ 
perceptions. 
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Table 5 


Students’ Drawings of Mathematicians 


aes pray Chi-squared Test 

Dimensions Items f (%) f (%) x p 
Gender 1. Male 8 (32) 5 (20) 0.93 33 

2. Female 6 (24) 5 (20) 0.11 13 

3. Undetermined 11 (44) 11 (44) 0.00 1,00 

4. No person - 4 (16) 3.19 07 

Expression 5. Smiling T (28) 10 (40) 0.80 37 
6. Frowning 6 (24) 2 (8) 2.38 12 

7. Facing backwards - 3 (12) 3.19 07 

Mathematical Symbols 8. Numbers 9 (36) 18 (72) 6.52 .01* 
9. Arithmetic 4 (16) 13 (62) 7.21 .001* 
10. Geometry 4 (16) 17 (68) 13.87 .001* 

11. None - - - - 

Instruments/Elements 12. Blackboard 7 (28) 12 (48) 2.12 14 
13. Desks 5 (20) 7 (48) 0.43 50 

14. Textbooks 6 (24) 2 (8) 2.38 12 

15. Writing materials 5 (20) 3 (12) 0.59 44 

16. Calculator/computer - - - - 

17. None - - - - 

Workplace 18. Indoors 16 (64) 10 (40) 2.88 89 
19. Outdoors 2 (8) 10 (40) 7.01 .001* 

20. Undefined T (28) 5 (20) 0.44 50 

21. Calculating/solving problems 14 (56) 10 (40) 1.28 25 

22. Explaining/teaching 8 (32) 5 (20) 0.93 33 
peton 23. Discovering/researching 2 (8) 40 (40) 7.01 001" 
24. No action 1 (4) - 1.02 31 

21. Other - 


In Table 5, findings from the students’ drawings of mathematicians are presented. There were no significant 
differences in the portrayals of gender or expressions between the pre-test and post-test. However, significant in- 
creases were observed in the portrayals of mathematical symbols including numbers (x? = 6.52, p= .01), arithmetic 
(x? = 7.21, p < .001), and geometry (x? = 13.87, p < .001). There were no significant differences in the instruments 
or elements portrayed in the drawings, but in the dimension of fields of work, a significant increase was seen in 
portrayals of the discovering/researching field (x? = 7.01, p < .001). A significant increase from 8% to 40% was also 
observed in portrayals of professionals engaged in discovering/researching (x? = 7.01, p < .001). 


Discussion 


The purpose of this research was to examine the changes in stereotypes held by female students regarding 
STEM professionals. With this aim, students were asked to draw pictures of STEM employees and their environ- 
ments and to explain their drawings. This allowed the students to reflect on changes in their views, personality 
traits, and practices related to STEM professionals before and after attending a STEM camp. The relevant literature 
suggests that girls are negatively affected by a lack of STEM knowledge and low self-esteem (Merayo & Ayuso, 
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2022; Pilotti, 2021). Therefore, in the present study, changes in female students’ perceptions of STEM professionals 
after attending workshops and career talks given by female STEM professionals were examined. The findings are 
discussed in more detail below in connection with the relevant literature. 

Students’ images of scientists have been of interest to researchers for nearly 50 years. Multiple research studies 
have demonstrated that when adolescents are tasked with drawing images of scientists or STEM professionals, their 
drawings typically embody stereotypes associated with these fields (Aguilar et al., 2016; Ates et al., 2020; DeWitt 
& Archer, 2015). Similarly, in the pre-test of the present research, it was seen that female students had fixed views 
of people working in scientific fields. However, stereotypical images may be changed with intervention programs, 
as both the present work and various past studies have found that intervention programs have a positive effect 
on changing students’ images of science (Emvalotis & Koutsianou, 2018; Karacam et al., 2021; Leblebicioglu et al., 
2020). Interactions with female STEM professionals positively contributed to an increase in portrayals of scientists 
as women in the present study. In addition, changes were observed in the portrayals of lab coats, people working 
indoors, and scientists as middle-aged or elderly people. These findings may be due to the fact that students had 
fixed thoughts and stereotypical images of scientists at the beginning of the study because they did not have any 
previous relationships with professionals (Pan et al., 2022) working in the fields of science. As a result of the STEM 
camp, female students’ ideas about the work of scientists matured. They showed conceptual development in their 
perceptions of scientists engaged in actions such as making things/working with hands, discovering/researching, 
and experimenting/testing. Although the present findings were obtained via informal education in a short period 
of time, it can be suggested that professionals should also interact with students in and out of the classroom in 
more formal applications. 

It is known that students have positive attitudes toward technology. However, their images of professionals 
working in the field of technology are limited and there has been insufficient research on this subject Jung & 
Kim, 2014). In the first stage of the present study, when the students drew pictures of individuals working in the 
field of technology, they generally portrayed people working at desks with computers, tablets, or phones and the 
people portrayed in these drawings were usually male. In the drawings from the post-test, however, portrayals of 
women working in the field of technology increased significantly. Portrayals of robots and drones increased in the 
post-test drawings. Thus, it can be concluded that the workshops conducted during the STEM camp affected the 
students’ perceptions of technologists. The students’ perceptions of workplaces in this field also changed over the 
course of the research, as their drawings shifted from portrayals of indoor environments to outdoor environments. 
To develop students’ rational and comprehensive understandings of technology in the context of effective STEM 
education, students must have an in-depth and accurate understanding of the nature of the different components 
of this professional field (Blom & Abrie, 2021). 

Similarly, in the field of technology, students have limited perceptions of engineering as a professional field 
(Fralick et al., 2009). In the present study, students generally drew figures of undetermined gender in the pre-test. 
Interest in engineering-related research has increased recently with the aim of changing gender stereotypes in 
perceptions of engineers. Female students tend to have limited perceptions of engineering professionals, and they 
are most likely to draw images of men wearing helmets and working indoors when asked to portray an engineer 
(Capobianco et al., 2011). Similar studies have found that students tend to portray engineers as people who do 
physical work, such as repairing and constructing, in pre-test drawings (Chou & Chen 2017). In the drawings pro- 
duced in the post-test session of the present study, it was seen that portrayals of female engineers working out- 
doors had increased. This supports previous claims that intervention programs are effective in changing students’ 
stereotypical (Kelly et al., 2017; Kuvac & Koc, 2022) and limited (NAE, 2008) perceptions of engineering. This may 
be achieved by balancing out male-oriented images (Fralick et al., 2009; Karatas et al., 2011) with the participation 
of female professionals in STEM programs (DeJarnette, 2012; Wyss et al., 2012). As a result of the STEM interven- 
tion program applied in the present study, in addition to a decrease in gender stereotypes, students’ perceptions 
of engineering-related work including actions such as repairs and project management were also strengthened. 
It can be concluded that STEM programs are effective in eliminating students’ misconceptions about engineer- 
ing. However, the lack of fixed courses or programs addressing science, technology, and mathematics remains an 
obstacle in many schools for improving students’ perceptions of the engineering profession. 

Itis widely agreed that men are more represented than women in mathematics as well as in science (Gjovik et 
al., 2022). Picker and Berry (2000) found that students’ portrayals of mathematicians involved bald or messy men. 
However, contrary to the literature, the findings obtained in the present study did not show a statistically signifi- 
cant difference in the gender representations of the students in their pre-test and post-test drawings. Students’ 
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images of people working in the field of mathematics tended to be vague. Rock and Shaw (2000) concluded from 
an analysis of students’ images of mathematics and mathematicians that the students perceived mathematics as 
an unattractive subject. However, in the present study, portrayals of the mathematical concepts of arithmetic and 
geometry increased. Since mathematics courses in formal school settings are often only concerned with problem- 
solving skills, students may not have enough knowledge about what mathematicians do. For example, in this study, 
they were also described as people who could work outdoors. Overall, gender representation in mathematics in 
STEM education remained uncertain, and the students’ knowledge of mathematics-related occupations was also 
limited. They lacked a realistic understanding of the people working in these fields. Overall, students’ impressions 
of scientists and technologists are typically superficial and misleading and may sometimes reflect ignorance (Mc- 
Creedy & Dierking, 2013; Scherz & Oren, 2006). Considering the results obtained here, it is possible to conclude 
that, in general, students see STEM fields as the realm of men, excluding mathematics (Martins et al., 2021), but 
this perception can be changed with the application of STEM intervention programs (Baran et al., 2019; Gonzalez- 
Pérez et al., 2020). This research is accompanied by a number of limitations. The study focused solely on female 
high school students, limiting the generalizability of the findings to other populations, such as male students or 
students from different age groups. The intervention examined was a STEM camp, which may not be representa- 
tive of all types of STEM programs and initiatives. The study did not include a control group, which would have 
allowed for a more rigorous assessment of the impact of the STEM camp on students’ perceptions compared to a 
group that did not participate in the intervention. 


Conclusions and Implications 

This study has shown that an informal STEM education program was effective in changing female students’ 
images of STEM professionals, in support of previous research. The students also generally found the unique 
features of this informal learning setting to be beneficial, as it provided the freedom to pursue their own interests 
and values, did not entail pressure related to academic requirements or standardized tests, and allowed them to 
work in collaborative groups. 

When the gender representations of STEM professionals were examined, students’ pre-test drawings were 
found to overwhelmingly represent men in the fields of science, technology, and engineering. The only exception 
was mathematics. However, in post-test drawings, their representations of female professionals in the fields of sci- 
ence, technology, and engineering numerically increased. Thus, interactions with female STEM role models seem 
to have had a positive impact on the students’ perceptions and STEM stereotypes can be changed in this way. 

The results of this research make a valuable contribution to the existing body of literature focused on female 
students’ perceptions of individuals working in STEM fields. It is evident that these perceptions can indeed be 
positively influenced through extracurricular educational initiatives. To foster a more accurate and positive un- 
derstanding of STEM professions among students, educators, policymakers, and researchers should be proactive 
in engaging students in STEM-related activities and providing opportunities for them to interact with a diverse 
range of STEM professionals. By doing so, we can cultivate more realistic and optimistic attitudes among students 
toward potential careers in STEM fields. 

Socio-cognitive career theory has generally not addressed important factors such as gender roles and gen- 
der norms. This research examined in more detail the gender-related factors that influence students’ interest and 
engagement in STEM fields. The research explored effective ways to change gender-related misconceptions and 
stereotypes, especially in STEM fields, and therefore provided recommendations for improving STEM education. 
The results of this study can help us understand the impact of programs designed to promote gender equality in 
STEM fields and contribute to developing strategies to combat gender inequality in STEM at the international level. 

The value of this research in an international context lies in contributing to a broader understanding of how 
educational interventions such as STEM camps and workshops led by female STEM professionals can influence 
students’ perceptions and stereotypes of STEM fields. The findings of the study can provide insights into how 
students’ perceptions of STEM professionals vary across different cultural and educational contexts. Although the 
research focused ona specific region or country, its methodologies and results can be compared with similar stud- 
ies in other parts of the world. This cross-cultural perspective can help identify common challenges and effective 
strategies for addressing gender stereotypes in STEM. The research highlights the effectiveness of informal STEM 
education, such as STEM camps and workshops, in changing students’ perceptions. These informal educational 
approaches can be adapted and applied to various international settings to promote diversity and inclusion in 
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STEM fields. Policymakers and educators worldwide can benefit from understanding the potential impact of such 
programs. Gender inequalities in STEM fields are a global problem. By revealing how interactions with female STEM 
professionals positively influence students’ perceptions, this study provides valuable insights into strategies for 
promoting gender equity in STEM education and careers. These insights can inform international efforts to close 
gender gaps in STEM. While the study focused on upper-secondary school female students in a specific context, its 
findings can serve as a basis for broader discussions and adaptations. Researchers and educators around the world 
can examine how the results align with their own experiences and explore the feasibility of implementing similar 
interventions in their own regions. This research can foster international collaboration among researchers, educa- 
tors and organizations interested in promoting diversity in STEM. The methods and results of the study can be a 
starting point for cross-border initiatives aimed at challenging stereotypes and promoting inclusion in STEM fields. 


Recommendations 


This research examined the impact of image-oriented information about STEM fields provided to female 
students in the course of a STEM camp. However, the participating students may have already had a high interest 
in pursuing careers in STEM fields as they were willing to participate in this educational camp. Therefore, future 
studies should be conducted with female students with different interest levels. In this study, changes in students’ 
STEM perceptions were examined immediately after the conclusion of a six-day STEM camp. Future studies may 
focus on the permanence of such effects. Finally, this study focused on a mixed-gender group of instructors work- 
ing with only female students. The question of whether different results would be achieved when female students 
work with only men or with only women should also be considered. 
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Introduction 


As societies have increasingly depended on science and technology, 
global educational systems have prioritized the mastery of science, technol- 
ogy, engineering, and mathematics (STEM) subjects. STEM education is a 
prerequisite for fostering innovation, driving industrialization, creating jobs, 
and fueling economic progress. Students’ proficiency in these disciplines 
indicates a nation’s educational quality (Kazu & Kurtoglu Yalcin, 2021; Strelan 
et al., 2020). 

The Trends in International Mathematics and Science Study (TIMSS) 
and Programme for International Student Assessment (PISA) are interna- 
tional assessment programs measuring students’academic performance. The 
TIMSS initiative, conducted once in four years starting in 1995, focuses on 
evaluating the academic achievements of eighth-grade students (typically 
15-year-olds) in mathematics and science. The number of countries in this 
assessment has risen from 46 to 60 in its latest iteration in 2019. However, 
the number of participating countries in the eighth-grade assessment has 
declined from 46 to 39 across the two cycles. On the other hand, the PISA, 
launched in 2000 and reiterated every three years, evaluates eighth-grade 
students’ proficiency in reading, mathematics, and science. It also gauges 
students’ capacity to apply acquired knowledge in practical, real-world 
scenarios. The number of countries participating in PISA has grown from 41 
initially to 78 in the 2018 cycle. 

Several studies on students’ performance in science have been con- 
ducted using data from TIMSS and PISA (Jiang et al., 2023; Kilig Depren, 2020; 
Lay & Chandrasegaran, 2018; Lay & Ng, 2021; Ozkan &Tekeli, 2021; Shannag et 
al., 2013; Tang et al., 2022). Some deal with international comparisons (Askin 
& Oz, 2020; Geske & Kangro, 2002; Lay & Chandrasegaran, 2018). 

This study uses TIMSS 2019 data to analyze the factors affecting the 
academic performance of eighth-grade students in Malaysia and Singapore to 
stimulate discourse on the improvements in science education and enhance 
equity based on empirical evidence. Both nations have invested substantially 
in education to cultivate human capital and drive economic growth. With a 
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Abstract. Absenteeism and students’ 
attitudes toward science have an impact 
on the exam results among eighth-grade 
students in Malaysia and Singapore. This 
study employed weighted least squares 
and quantile regression techniques on 

the Trends in International Mathematics 
and Science Study (TIMSS) 2019 data 

to analyze the net effect of these two 
factors in influencing the academic 
performance of eighth-grade students. 
Given Singapore's considerable lead in 
TIMSS ranking over Malaysia, this research 
sheds light on modifiable factors that 

can help Malaysia enhance its national 
science education. Absenteeism was more 
pressing in Malaysia than in Singapore, 
but it significantly influenced exam results 
in both countries. However, there was no 
significant difference in attitudes toward 
science. Results from quantile regression 
show that these impacts were not uniform 
across the performance distributions. The 
study's results emphasize that absenteeism, 
home educational resources, and parental 
education contribute to the performance 
disparities between the two countries. 
These results underscore the importance 
for policymakers and educational planners 
in Malaysia to take proactive steps in 
addressing these shortcomings to improve 
students’ exam results, to work towards 
narrowing the performance gap, and to 
enhance science education. 
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mean science score of 608, Singapore's students have been ranked consistently as the top performers, while Ma- 
laysia with a mean science score of 460 was placed at 29 out of 39 participating countries in 2019. Hence, explor- 
ing the gross and net effects of factors influencing science performance in both countries can provide insights to 
guide Malaysia in improving its national science education. While there was a comparative study in Malaysia and 
Singapore on teacher preparation on students’ science achievement (Lay & Chandrasegaran, 2018) and one on 
the comparison of school climate on science achievement among South-East and East Asian countries (Lay & Ng, 
2021), to our knowledge, there is no study comparing the impacts of absenteeism and students’ attitudes toward 
science on exam results among students in the two neighboring countries. 


Literature Review 


A multitude of factors influence students’ science achievement. There is ample evidence that absenteeism 
and students’ attitudes toward science are among the key factors influencing their science achievement (Akkus 
& Cinkir, 2022; Arulampalam et al., 2012; Keppens, 2023; Klein et al., 2022; London et al., 2016). These studies sup- 
port the faucet theory, which posits that frequent exposure to schooling improves students’ skills, and the school 
disengagement theory on why absenteeism might lead to lower academic achievement (Alexander et al., 2001; 
Keppens, 2023). 

Absenteeism is associated with the socio-economic background of students. One study has found that stu- 
dents with a disability and the impoverished were the most likely to have missed school (Garcia & Weiss, 2018). 
Several studies have found that the main reasons for skipping classes include poverty, lack of parental guidance 
and support, lack of interest, unconducive classrooms, poor teacher approach, substance abuse, punishment at 
school, ineffective Parent Teacher Association, and the mushrooming of entertainment and video centers (Center 
for Research in Education and Social Policy, 2018; Garcia & Weiss, 2018; Moonie et al., 2008). Student absenteeism 
also challenges teachers as they have difficulty in classroom management and waste time repeating lessons for 
the absent students (Akkus & Cinkir, 2022). 

The significance of students’ attitudes toward science cannot be overstated, as these attitudes shape their 
engagement, motivation, and dedication to science-related activities. These attitudes and behaviors can either 
facilitate or hinder the learning process, which affects their science performance. The underpinning theoretical 
rationale for the importance of attitudes toward STEM subjects finds support in various theories encompassing edu- 
cational, social, and vocational psychology. Notably, Eccles’expectancy-value theory provides a robust framework, 
positing that attitudes, including perceptions of success and task value, alongside self-concept, greatly influence 
educational decisions (Else-Quest et al., 2013). 

The impact of positive attitudes toward STEM subjects extends to creating an enriching and receptive learning 
environment. Conversely, negative attitudes can culminate in disinterest and suboptimal learning outcomes. The 
correlation between attitudes and achievements in science has recently garnered heightened attention within the 
research community. Numerous investigations have unveiled a positive association between favorable attitudes 
toward science and exam results, consistent with hypotheses (Al-Mutawah & Fateel, 2018; Ali et al., 2015; Askin & 
Oz, 2020; Chang & Cheng, 2008; Else-Quest et al., 2013; Mao et al., 2021). Juan Andrea et al. (2014) have found that 
all three science attitudinal indices of enjoyment, valuing science, and self-confidence in learning science have 
contributed significantly to grade 9 learners’ science scores. However, results from a meta-analysis have shown 
that the type of attitude has moderated a positive correlation between attitudes toward science and learning 
outcomes (Mao et al., 2021). 

Several studies also found a positive association between self-efficacy and science achievement (Askin & Oz, 
2020; Juan et al., 2018; Kirbulut & Uzuntiryaki-Kondakci, 2019; Lam & Lau, 2014; Larson et al., 2013). Students who 
are confident in science tend to do well in the subject. Louis and Mistele (2012) have found a statistically significant 
gender difference among science subjects after controlling for self-efficacy. Besides, students with higher educa- 
tional goals tend to perform better than less ambitious students (Askin & Oz, 2020). These findings call for more 
intervention strategies on non-cognitive aspects to enhance science learning. 

Absenteeism is often a symptom of broader socio-economic challenges (Sosu et al., 2021). When exploring the 
repercussions of absenteeism and students’ attitudes toward science on their academic achievements, it becomes 
evident that factors such as family background, socio-economic status, parental education, parental involvement, 
and home educational resources also wield substantial influence over students’ educational outcomes, which must 
be considered. Family background and socio-economic status profoundly impact a student's access to resources, 
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opportunities, home environment, and support networks. Research indicates that family scholarly culture, encom- 
passing parents’ education and the number of books in the home, correlates positively with educational success 
across diverse economic contexts (Askin & Oz, 2020; Evans et al., 2010). 

Students from varying socio-economic backgrounds often encounter disparities in exposure to educational 
materials, technology, and enriching extracurricular activities. These discrepancies can subsequently impact their 
engagement with science and overall academic performance, as evidenced by prior studies (Ferguson et al., 
2007; Fergusson et al., 2008; Khan et al., 2019; Li et al., 2020; Li & Qiu, 2018; Sirin, 2005). Instances also arise where 
students miss classes due to illness or to assist with domestic responsibilities and their parents’ work (Keppens, 
2023; Klein et al., 2022). 

Parental education and involvement emerge as critical determinants that can either enhance or impede a 
student's affinity for science and attendance at educational institutions. Parents with higher educational levels are 
more likely to recognize the significance of a solid scientific foundation and can facilitate their children’s learning 
experiences. Furthermore, positive role models and ongoing discussions about the value of education can cul- 
tivate a culture of learning within the household, reinforcing a student’s commitment to school attendance and 
excellence in science-related subjects. Several studies highlight that the higher educational levels of both parents 
have positively correlated with their children’s academic achievements (Abu Bakar et al., 2017; Davis-Kean, 2005; 
Idris et al., 2020; Murnane et al., 1981). Intriguingly, one study has noted that while mothers’ education positively 
impacts students’ academic outcomes, the reverse is true for working women, except those who are teachers 
(Hoque et al., 2017). 

Furthermore, the depth of parental involvement, characterized by interactions with teachers, encouragement 
of curiosity, and provision of learning resources at home, has significantly molded students’ motivation to embrace 
science and sustain consistent engagement in their studies. Research has demonstrated that parental involvement 
positively impacts students’ academic performance (Fan & Chen, 2001; Lara & Saracostti, 2019). 

School resources have also been found to affect students’ exam results (Hanushek, 1997; Ndlovu, 2018). 
Graddy and Stevens (2005) found that the student-teacher ratio at a school is negatively correlated with students’ 
exam results. Moreover, school resources have been found to reduce the socio-economic achievement gap in PISA 
(Yang & Lee, 2022). 


Research Problem 


Beyond imparting knowledge about the natural world, science education nurtures vital skills like critical 
thinking, problem-solving, and analytical reasoning. The emphasis on STEM education is typically initiated at the 
secondary educational level because this stage is crucial in determining a student's academic trajectory. 

Given the importance of STEM education, there is a need for a comprehensive assessment of students’ 
achievements in these subjects and their influencing factors. By understanding the factors that impact science 
performance, educational policymakers and stakeholders can identify areas for improvement in their respective 
education systems. This knowledge could lead to targeted interventions that enhance the quality of science edu- 
cation and subsequently improve students’ scientific literacy and competency. 

Many studies have analyzed the impacts of absenteeism and students’ attitudes toward science on aca- 
demic performance; only a few have utilized a nationally representative sample. Because chronic absenteeism 
is not well researched (London et al., 2016), this study examines the impact of the frequency of class absence on 
academic performance to address this gap. Moreover, the effects of absenteeism on performance may differ by 
gender, socio-economic status, and student ability (Arulampalam et al., 2012; Mooney et al., 2023). Using quantile 
regression analysis, Arulampalam et al. (2012) have found that absenteeism’s effects on academic performance 
are much more pronounced among better-performing students. There needs to be more research addressing the 
multifaceted impacts of various factors on the diverse range of examination outcomes. Hence, this study enhances 
the methodological toolkit for investigating educational disparities by incorporating weighted least squares and 
quantile regression methods. 


Research Focus 


This analysis primarily focuses on the impacts of absenteeism and students’ attitudes toward science on exam 
results. These factors are central due to their direct influence on engagement, measurability, potential for interven- 
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tions, alignment with educational goals, research support, and contribution to a comprehensive understanding 
of science education. Moreover, absenteeism is widely recognized as a significant factor associated with poorer 
academic outcomes, especially in science subjects, which typically involve hands-on experiments, interactive 
discussions, and practical sessions. Also, science subjects are cumulative, where concepts build upon each other, 
and hence, missing even a single class can make catching up difficult. 

The analysis of the impacts of absenteeism and students’ attitudes holds the potential to refine theoretical 
frameworks for explaining variations in exam results. It is essential to emphasize that this analysis considers sev- 
eral other factors commonly considered in educational research, including but not limited to home educational 
resources, parents’ educational levels, and school resources. These factors are primarily included as control variables 
to ascertain the net effect of absenteeism and students’ attitudes toward science on the exam results. 


Research Aim 


The study aimed to assess the impact of absenteeism and students’ attitudes toward science on exam per- 
formance across two countries characterized by a significant performance gap. The analysis has the potential to 
make substantial contributions to the existing literature, enrich educational theories, and guide evidence-based 
policies for enhancing educational outcomes and narrowing disparities. Examining two countries with a notable 
gap in science performance adds a novel dimension to the literature. Findings from this study could shed light on 
factors contributing to divergent outcomes, offering insights applicable to other educational contexts. 


Research Methodology 
General Background 


The data for this study were extracted from the most recent iteration of the TIMSS, conducted in 2019 in Malaysia 
and Singapore. TIMSS, overseen by the International Association for the Evaluation of Educational Achievement 
(IEA)’s TIMSS & PIRLS International Study Center at Boston College, is a widely acknowledged assessment. Follow- 
ing international protocols and TIMSS’ standards of technical excellence, each participating country appointed a 
National Research Coordinator responsible for supervising the execution of TIMSS. In conducting TIMSS, IEA has 
adopted various quality control mechanisms to ensure high-quality data that are valid and reliable (Wagemaker, 
2020). This study is based on large samples in Malaysia and Singapore, and the surveys were conducted using 
scientific methods that have been tested in many countries and over many cycles. 


Sample 


Each country employed a two-stage stratified sampling design to ensure a representative sample of schools 
and students on a national scale. The initial sampling phase encompassed the selection of schools based on prob- 
ability proportional to their size from a list of eligible schools. In Malaysia, stratification involved school type, score 
level within the Ministry of Education (M.O.E.) daily school strata, and urbanization (rural or urban) within all M.O.E. 
strata. In contrast, being entirely urban, Singapore did not require such stratification. A total of 177 schools were 
chosen in Malaysia, while Singapore selected 153. 

The subsequent sampling stage involved selecting intact classes from the eighth grade of each chosen school 
through systematic sampling, maintaining an equal probability of selection. Each National Research Coordina- 
tor used the Within-School Sampling Software (WinW3S), developed by IEA Hamburg and Statistics Canada, to 
facilitate class sampling. All students within the selected classes participated in the assessment. In Malaysia, 7,065 
students completed the assessment, resulting in a student participation rate of 98%. Meanwhile, 4,853 students 
in Singapore completed the assessment, with a student participation rate of 96%. A detailed explanation of the 
methods and procedures is available from TIMSS’ technical report (Martin et al., 2020). 


Instrument and Procedures 


For comprehensive data collection, four distinct sets of questionnaires were prepared: the student question- 
naire, teacher questionnaire, school questionnaire, and curriculum questionnaire. These questionnaires were 
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completed by the eighth-grade students and their parents, teachers, and school principals. The student question- 
naire, serving as the basis for this analysis, comprises 27 main questions, each containing various sub-questions, 
yielding 131 questions/sub-questions. 


Data Analysis 


This study's primary dependent variable was the science scores attained by eighth-grade students, assessed 
using a ratio scale. The TIMSS science scores were constructed using the mean of the five plausible values, repre- 
senting the proficiency variables drawn from the resulting posterior distribution for each student in each cognitive 
domain. These scores ranged from a minimum of 133.2 in Malaysia to a maximum of 824.4 in Singapore. 

The two independent variables for this analysis were the “Frequency of absence from school” and “Students 
value science.’ Absenteeism refers to a student being absent from school for various reasons such as lack of mo- 
tivation to study, ill health, having to work during school hours and taking care of sick family members, and poor 
relationships with other students. The frequency of absenteeism was coded as 1 = Once a week; 2 = Once every 
two weeks; 3 = Once a month; 4 = Once every two months; and 5 = Never or almost never. “Students value science” 
was coded as 1 = Strongly value science, 2 = Somewhat value science, 3 = Do not value science. This variable was 
derived from responses to nine Likert-scale questions regarding students’ attitudes toward valuing science. The 
other independent variables in this analysis were home educational resources, parents’ highest educational level, 
and “affected by science resource shortage.’ Home educational resources variable was coded as 1 = Many, 2 = 
Some, and 3 = Few. Parents’ highest educational level was coded as 0 = Do not know, 1 = University or higher, 2 = 
Post-secondary, 3 = Upper-secondary, 4 = Lower-secondary, and 5 = Some primary or none. “Affected by science 
resource shortage” was coded as 1 = Not affected, 2 = Affected, and 3 = Affected a lot. 

Two attitudinal variables, “Students like learning science” and “Students confident in science,’ are closely cor- 
related with “Students value science” and hence were not included in the analysis for the sake of parsimony. Contrary 
to expectation, an initial examination of the data revealed that the variables “Time spent on science subjects” and 
attendance of extra classes do not exhibit significant correlations with science scores, and the signs are not always 
consistent and are therefore not included in the analysis. 

All analyses were performed using SPSS version 29. Data were weighted using the overall sampling weight 
to ensure the findings accurately represented the population of eighth-grade students in each respective country. 
The overall sampling weight for each student was the product of the three weighting components: school, class 
(within school), and student (within class). 

The analysis began by presenting the mean science scores categorized across different independent variables. 
Subsequently, weighted least squares regression analysis was conducted to investigate the relationships between 
the independent variables and science exam results before and after accounting for other variables in the model. 
The weighted least squares regression coefficients were employed to quantify the magnitude and direction of 
these relationships. The investigation encompassed five distinct regression models, each focused on Malaysia and 
Singapore separately. The first two models individually evaluated the influence of school absence frequency and 
students’ valuing of science on the exam results. The third model examined the combined impact of these two factors 
while controlling for one another. Additionally, Model Four introduced home educational resources and parents’ 
educational levels as supplementary controls. In the fifth model, the effects of each independent variable were 
assessed while considering all other variables in the model, including the presence of science resource shortages. 

Moreover, quantile regression was independently conducted for Malaysia and Singapore to provide insights 
into the conditional distribution of the exam score at various quantiles. This approach aimed to estimate the con- 
ditional quantiles of exam results based on the predictor variables. As the effects of predictors can differ across 
various score ranges, quantile regression provides a more comprehensive understanding of how absenteeism, 
valuing science, and other predictors influence students at different performance levels. This analysis strategy 
allows for a nuanced comprehension of the relationships in situations where conventional ordinary least squares 
regression might not fully capture the variation across performance levels. 

The significance of the weighted least squares and quantile regression coefficients was based on the t-test. 
The cut-off value used to evaluate the significance is p < .05. 
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Research Results 
Bivariate Relationship between the Independent Variables and the Exam Scores 


Table 1 presents the distribution of samples of the key study variables in Malaysia and Singapore. It also shows 
the mean score in the science subject and the standard errors for these estimates. The data indicates a notable 
difference between Malaysian and Singaporean students concerning class attendance - Malaysian students were 
significantly more likely to be absent than their Singaporean counterparts. Specifically, among the Malaysian 
students in this study, as high as 17% were absent from class at least once a week (or chronic absence), compared 
to only 2.3% among the Singaporean students. Conversely, 74.9% of Singaporean students were never or rarely 
absent from class, compared to 40.2% of Malaysian students. 

In both countries, absenteeism significantly impacts students’ performance in science. In Malaysia, the average 
score ranged from 386 among students absent at least once a week to 492 among those who were hardly ever 
absent or never absent. In Singapore, the corresponding figures were 476 and 620 respectively. 

Most students in both countries strongly value science, with a notable proportion holding strong positive 
attitudes toward the subject (Table 1). Students valuing science is positively associated with their exam results, as 
evidenced by the science scores, ranging from 384 to 492 in Malaysia and 557 to 632 in Singapore. 

Singaporean students are better equipped with home educational resources than Malaysian students. Spe- 
cifically, 14.3% of Singaporean students have access to many home educational resources, whereas only 4.4% of 
Malaysian students enjoy the same resources. Conversely, relatively more Malaysian students, 20.5%, have limited 
access to educational resources at home, while only 8.1% of Singaporean students face a similar situation. 

The availability of home educational resources significantly impacts students’ science achievement in both 
countries. In Malaysia, students with limited access to such resources obtained an average score of 413, while 
those with ample resources at home scored much higher, averaging 549. Similarly, in Singapore, students with 
few home educational resources achieved an average score of 525, whereas those with abundant resources scored 
significantly higher, averaging 659. 

Among the surveyed students, 65% to 70% reported their parents’ educational levels. The data revealed that 
Singaporean parents generally had a higher education than Malaysian parents. Notably, 51% of Singaporean parents 
held tertiary education degrees, while only 15% of Malaysian parents achieved the same level of education (among 
those who reported). In Malaysia, the most common educational level among parents was upper-secondary and 
post-secondary education, which constituted 65% of the surveyed population. 

The impact of parents’educational attainment on students’ science achievement was evident from the results. 
There was a significant gap of over 100 points in the science scores between students whose parents had some 
primary education and those whose parents had tertiary education — ranging from 416 to 531 in Malaysia, and 
from 524 to 648 in Singapore. 

Regarding school science resource availability, Malaysian schools are significantly more prone to facing short- 
ages than their Singaporean counterparts. Nearly all schools in Malaysia grapple with science resource shortages; 
a substantial 20.4% experience severe shortages. In contrast, only a quarter of schools in Singapore encounter 
resource shortages, with less than 7% facing acute shortages. 

Surprisingly, despite these disparities in resource availability, this factor did not have a discernible and con- 
sistent effect on students’ science achievement in both countries. The impact of resource shortages on academic 
performance in science subjects was not evident, suggesting that other factors might be at play in shaping students’ 
achievements in this area. 

It is essential to acknowledge that the estimates presented here were based on a large sample size, which 
resulted in very small standard errors. This fact strengthens the reliability and robustness of the findings. However, 
it is worth mentioning that there were a few categories within the independent variables with relatively smaller 
sample sizes in Singapore. These categories included students absent at least once a month, parents with low 
educational levels, and schools facing acute resource shortages. Due to the smaller sample sizes, these specific 
categories exhibited relatively larger standard errors in their estimates. Nonetheless, these standard errors did not 
exceed 3.2 even in the worst-case scenario, indicating that the estimates remain reasonably stable and dependable. 
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Table 1 


ABSENTEEISM AND STUDENTS’ 


ATTITUDES TOWARD SCIENCE: IMPACT ON EXAM RESULTS 
AMONG EIGHTH-GRADE STUDENTS IN MALAYSIA AND SINGAPORE 


Malaysia (n = 7,065) 


Sample Characteristics, Mean Science Score, Standard Error of the Mean, Malaysia and Singapore 


(pP.991-1010) 


Singapore (n = 4,853) 


oe Sample X SE Sample X SE 
distribution (%) distribution (%) 

Frequency of absence 
Once a week 17.0 386.48 0.30 2.3 476.14 3.02 
Once every 2 weeks 10.3 433.84 0.40 3.0 535.90 2.61 
Once a month 14.3 460.05 0.34 6.7 563.18 1.72 
Once every 2 months 18.3 474.91 0.30 13.1 597.70 1.26 
Never or almost never 40.2 492.46 0.22 74.9 620.43 0.45 
Students value science 
Strongly value science 44.8 492.41 0.19 41.6 632.04 0.60 
Somewhat value science 46.2 445.31 0.21 47.6 597.61 0.63 
Do not value science 9.0 383.67 0.48 10.8 557.44 1.29 
Home educational resources 
Many 44 548.76 0.54 14.3 658.70 0.82 
Some 75.0 468.53 0.16 776 606.82 0.47 
Few 20.5 412.54 0.30 8.1 524.58 1.65 
Parents’ education 
Don’t know 35.0 440.93 0.25 30.0 591.55 0.80 
University or higher 9.5 531.14 0.39 35.7 648.21 0.56 
Post-secondary 16.4 491.60 0.33 20.3 588.43 0.94 
Upper-secondary 25.9 463.27 0.27 8.9 583.25 1.34 
Lower-secondary 8.1 424.82 0.48 2.6 548.96 3.17 
Some primary or none 5.2 416.13 0.56 2.5 524.33 2.93 
Science resource shortage 
Not affected 2.3 490.93 1.31 75.4 607.31 0.51 
Affected 77.3 454.18 0.16 17.9 606.78 0.87 
Affected a lot 20.4 478.91 0.33 6.7 612.35 1.81 


Note: Missing values are excluded from the calculations. 


Table 2 presents data on the percentage of Malaysian and Singaporean students who skipped classes at least 


once a month, categorized by various factors. Notably, Malaysian students exhibited a strikingly higher tendency 


to skip classes, surpassing their Singaporean counterparts by more than threefold (41.6% compared to 12%). 
Within Malaysia, a substantial gender disparity emerged, with boys showing a significantly greater propensity to 


skip classes than girls. However, in Singapore, this gender divide in class attendance was not statistically significant. 


Across both countries, the frequency of class skipping exhibited significant associations with several key fac- 
tors: students’ lack of appreciation for science, absence of aspirations for higher education, and limited access to 
educational resources and support at home. Furthermore, students from families with lower parental educational 
levels and those residing in economically disadvantaged areas were found to be more prone to skip classes. 
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Percent of Students Who Were Absent At Least Once a Month in Malaysia and Singapore 


Variables Malaysia (n = 7,065) Singapore (n = 4,853) 

Overall 41.6 12.0 

Gender 

Girl 36.7 12.2 

Boy 46.6 11.9 
Students value science 

Strongly value science 34.8 10.0 
Somewhat value science 46.6 12.9 

Do not value science 47.7 15.8 


How far in education one expects to go 


Secondary education 54.8 26.9 
Post-secondary, non-tertiary education 49.2 28.6 
Short-cycle tertiary education 43.0 16.5 
Bachelor’s or equivalent level 30.7 8.8 
Postgraduate degree 26.8 8.9 
Home educational resources 

Many 20.1 6.1 

Some 39.8 11.8 
Few 52.5 24.6 
Home educational support 

Neither own room nor internet connection 47.9 23.2 
Either own room or internet connection 43.5 13.8 
Both 39.1 10.7 
Parents’ education 

Don’t know 46.0 11.8 
University or higher 27.2 8.2 
Post-secondary 33.1 13.9 
Upper-secondary 42.9 16.7 
Lower-secondary 49.8 23.6 
Some primary or none 43.7 25.8 
Affluence 

0 to 10% 47.8 17.3 
11 to 25% 40.0 11.1 
26 to 50% 31.1 9.4 
More than 50% 21.2 9.5 
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Results from Weighted Least Squares Regressions 


The weighted least squares regression results on the Malaysian students revealed the following findings: 
Absenteeism alone accounted for 16.9% of the variance in science scores. In comparison, students’ valuing of sci- 
ence explained 12.6% of the variance (Table 3). They explained 26.1% of the variance when both variables were 
considered together. However, with the addition of home educational resources and parents’ educational level 
to the model, the adjusted coefficient of determination increased substantially to 33.3%. Interestingly, including 
school science resource shortage increased the explanatory power only slightly to 34.2%. 

Upon further investigation, adjusting for students’ valuing of science reduced the gap between those who 
were absent at least once a week and those who were hardly or never absent slightly, from about 106 points to 97 
points—including home educational resources and parents’ educational level in the model further narrowed the 
gap to 85 points. The addition of school resource shortage had a negligible effect on the gap. 

Similarly, the gap between students who did not value science and those who strongly valued it diminished 
from 109 points to 96 points after adjusting for absenteeism. Including home educational resources and parents’ 
educational levels in the model decreased the gap to 84 points. Once again, the school science resource shortage 
did not impact the gap. 

Among the other independent variables, home educational resources and parents’ educational level emerged 
as significant predictors of students’ science achievement. Students with access to abundant educational resources 
scored about 61 points higher than those with limited resources, even after controlling for absenteeism, valuing 
science, and parents’ educational level. Additionally, students whose parents had attained tertiary education scored 
57 points higher than those whose parents had only completed primary education, even after accounting for other 
variables in the model. However, school science resource shortage did not have a consistent effect on the exam 
results. Therefore, in the interest of simplicity and conciseness, Model 4 best elucidates the factors influencing the 
science exam results among eighth-grade students. 


Table 3 
Weighted Least Squares Regression Coefficients (Standard Errors), Malaysia (n = 7,065) 


Variables Model 1 Model 2 Model 3 Model 4 Model 5 
Frequency of absence 
Once a week -105.96*** -96.68*** -85.22*** -84.32*** 
(0.38) (0.37) (0.36) (0.36) 
Once every 2 weeks -58.60*** -54.78*** -48.08*** -47.28*** 
(0.46) (0.44) (0.43) (0.42) 
Once a month -32.39*** -31.30** -28.47*** -27.61*** 
(0.41) (0.39) (0.37) (0.37) 
Once every 2 months -17.53*** -18.76*** -17.63*** -17.42*** 
(0.38) (0.36) (0.34) (0.34) 


Never or almost never (ref) 
Students value science 


Strongly value science 108.72*** 96.43*** 84.44*** 85.14%" 
(0.50) (0.46) (0.45) (0.45) 

Somewhat value science 61.63*** 58.79" 53.32*** 53.47" 
(0.50) (0.46) (0.44) (0.44) 


Do not value science (ref) 
Home educational resources 


Many 61.41" 59.60*** 
(0.73) (0.73) 

Some 26.53*** 26.23*** 
(0.36) (0.36) 
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Variables Model 1 Model 2 Model 3 Model 4 Model 5 
Few (ref) 
Parents’ education 
Don’t know 10.09*** 9.43*** 
(0.63) (0.62) 
University or higher 57.32*** 55.31*** 
(0.76) (0.75) 
Post-secondary 34.74*** 33:33" 
(0.69) (0.69) 
Upper-secondary 22.34*** 21.05*** 
(0.65) (0.64) 
Lower-secondary 2.43*** 1.95** 
(0.69) (0.69) 


Some primary or none (ref) 
Science resource shortage 


Not affected 13.69*** 
(0.85) 

Affected -20.12*** 
(0.30) 


Affected a lot (ref) 


Constant 492.46** 383.70°* 420.57 382.52** 398.39 
(0.21) (0.46) (0.45) (0.68) (0.72) 

R? 169 126 262 333 342 

Adj. R? 169 126 261 333 342 

F test statistic 20,523.60 28,909.12 23,557.05 15,091.57 13,583.38 

(p) <.001 <.001 <.001 < 001 <.001 


Note: ***p < .001, **p < .01 


In Singapore, absenteeism accounts for 11.3% of the variance, while students’ valuing of science explains 
7.9%. Combining these two factors explains 18.4% of the variance in science scores. Adding students’ access to 
home educational resources and parents’ education to the model raises the adjusted coefficient of determination 
to 30.2%. However, adding school science resource shortage does not contribute any explanatory power to the 
model (Table 4). Itis worth noting that the percentages of variance explained by the variables in Singapore are lower 
than those in Malaysia, partly due to a smaller sample size and a skewed distribution in the independent variables. 

The impact of absenteeism on science achievement remains relatively consistent even when considering 
students’ valuing of science. For instance, the disparity in science scores between those frequently absent (at least 
once a week) and those rarely or never absent would only slightly diminish from 144 points to 141 points. However, 
the influence of absenteeism on science achievement is significantly reduced when considering students’ access 
to home educational resources and parents’ education. In this case, students who were absent weekly scored 118 
points lower than those who were rarely or never absent from class. 

Students’ perception of the science subject is also significant in determining their exam results. If there were 
no difference in absenteeism, students who highly value science would have scored 71 points higher than those 
who do not value it, a slight reduction from the observed 75-point difference. Moreover, when considering ac- 
cess to home educational resources and parents’ educational level, this difference is further reduced to 56 points. 

Access to home educational resources and parents’ educational achievement are also significant predictors 
of students’ science achievement. After controlling for absenteeism, students’ valuing of science, and parents’ 
educational level, students with access to abundant resources would have scored 74 points higher than those who 
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had limited resources. Additionally, students whose parents had tertiary education would have achieved 48 points 
higher than those with only primary education, even after adjusting for other variables in the model. The short- 
age of school science resources does not influence students’ exam results, as indicated by its lack of a significant 
association with exam performance. Furthermore, it plays no part in improving the model’s explanatory power 
or moderating the effects of other variables on the exam results. Hence, Model 4 is the best model for this study. 


Table 4 
Weighted Least Squares Regression Coefficients (Standard Errors), Singapore (n = 4,853) 


Variables Model 1 Model 2 Model 3 Model 4 Model 5 
Frequency of absence 
Once a week -144.48*** -141.08*** -118.10*** -117.96"** 
(2.75) 2.64) (2.48) (2.48) 
Once every 2 weeks -84.35*** -81.47*** -63.62*** -63.52*** 
(2.39) 2.30) (2.15) (2.15) 
Once a month -57.23*** -52.41*** -47.15*** -47.16*** 
(1.64) 1.58) (1.47) (1.47) 
Once every 2 months -22.74*** -22.38*** -19.52*** -19.43*** 
(1.22) 1.17) (1.09) (1.09) 


Never or almost never (ref) 
Students value science 


Strongly value science 74.67%** 70.88** 56.19*** 56.31*** 
(1.42) (1.34) (1.26) (1.26) 

Somewhat value science 40.14** 38.73" 32.31% 32.44%" 
(1.40) (1.32) (1.23) (1.23) 


Do not value science (ref) 
Home educational resources 


Many 73.83*** 73.74°* 
(1.96) (1.96) 

Some 51.98*** 51.80*** 
(1.62) (1.63) 

Few (ref) 

Parents’ education 

Don’t know 11.70*** 11.92*** 
(2.66) (2.66) 

University or higher 47.81*** 48.11*** 
(2.76) (2.76) 

Post-secondary 4.62 4.81 
(2.78) (2.78) 

Upper-secondary 10.60*** 10.74*** 
(2.83) (2.83) 

Lower-secondary 2.40 2.77 
(3.24) (3.24) 


Some primary or none (ref) 
Science resource shortage 


Not affected -1,.54 
(1.47) 
Affected 1.28 
(1.65) 
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Affected a lot (ref) 


Constant 620.41** 557.43 571.98** 505.96** 506.68** 
(0.47) (1.26) (1.23) (2.53) (2.85) 

R? 113 079 184 302 302 

Adj. R? 113 079 184 302 302 

F test statistic 1,224.57 1,652.72 1,435.24 1,270.33 1,101.78 

(p) <.001 <.001 <.001 <.001 <.001 


Note: ***p < .001 


Results from Quantile Regressions 


Absenteeism significantly adversely impacted science exam performance among Malaysian students across 
all quantiles, being more pronounced at higher quantiles. For instance, at the q10 quantile, students who skipped 
classes at least once a week scored 77.9 points lower than their peers who rarely or never missed classes after 
controlling all other variables in the model. This gap widened in the q75 and q90 quantiles, with differences of 
89.4 and 88.7 points, respectively, compared to the reference group (Table 5). 

Students’ perception of science also significantly impacted their science scores, which had a more pronounced 
effect at the lower quantiles than at higher quantiles. At q10, those who strongly valued science scored 101.7 points 
higher than those who did not value science after accounting for the influence of other variables. However, this 
difference diminished to 62.9 points at q90. 

Home educational resources and parents’ education also significantly impacted science performance, par- 
ticularly at the lower quantiles. For example, those with access to abundant resources scored 73.9 points higher 
at q10, decreasing to 53.6 points at q90. Similarly, students whose parents had university education scored 71.5 
points higher than those whose parents had only some primary education at q10, but the gap reduced to 44.8 
points at q90. 

Students from schools unaffected by science resource shortages performed significantly better than those 
facing shortages across all quantiles except q25. However, it is worth noting that the disparity in science perfor- 
mance between those from schools severely affected by resource shortages and those from schools moderately 
affected was contrary to expectations. 


Table 5 
Quantile Regression Coefficients (Standard Errors), Malaysia (n = 7,065) 


Variables qi0 q25 q50 q75 q90 

Frequency of absence 

Once a week -77.90** -85.38°* -87.37*"* -89.41*** -88.66"** 
(0.74) (0.51) (0.44) (0.46) (0.51) 

Once every 2 weeks -50.34*** -43.49*** -48.93*** -49.75** -56.78*** 
(0.88) (0.61) (0.52) (0.54) (0.60) 

Once a month -25.08*** -26.73** -24.35*** -35.15"* -37.62*** 
(0.77) (0.53) (0.45) (0.47) (0.52) 

Once every 2 months -11.37** -13.21°"* -15.94*** -23.84*** -20.35*** 
(0.70) (0.49) (0.42) (0.43) (0.48) 


Never or almost never (ref) 

Students value science 

Strongly value science 101.68*** 95.11*** 86.92*** i3.foe? 62.93*** 
(0.93) (0.64) (0.55) (0.57) (0.63) 
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Variables qi0 q25 q50 q75 q90 
Somewhat value science 56.75*** 55.96*** 55.89*** 49.00*** 43.42** 
(0.91) (0.63) (0.54) (0.56) (0.62) 


Do not value science (ref) 
Home educational resources 


Many 73.87*** 62.85*** 50.45*** 51.60*** 53.55*** 
(1.50) (1.04) (0.89) (0.93) (1.02) 

Some 31.13*** 26.28*** 19.67*** 24.35*** 26.12*** 
(0.75) (0.52) (0.44) (0.46) (0.51) 

Few (ref) 

Parents’ education 

Don’t know 9.04*** L12"* 17.70*** 6.44*** 9.91*** 
(1.29) 0.89) (0.76) (0.79) (0.88) 

University or higher 71.45*** 60.34*** 64.87*** 45.70*** 44,.75*** 
(1.56) 1.08) (0.92) 0.96) (1.06) 

Post-secondary 33.68*** 36.81*** 45.66*** 32.16*** 27.80*** 
(1.43) 0.99) (0.84) 0.88) (0.97) 

Upper-secondary 28.34*** 22.45*** 29.64*** 18.48*** 16.33*** 
(1.33) 0.92) (0.79) 0.82) (0.91) 

Lower-secondary 3.14* 241" 9.64*** -3,.23*** 1.94* 
(1.42) (0.99) (0.84) 0.88) (0.97) 


Some primary or none (ref) 
Science resource shortage 


Not affected 4.18* 7.88 16.07 27.34*** 53.04*** 
(1.75) (1.21) (1.04) (1.08) (1.19) 

Affected -13.00** -18.88*** -20.33*** -20.23*** -20.51** 
(0.63) (0.44) (0.37) (0.39) (0.43) 


Affected a lot (ref) 


Constant 275.08** 341.72" 396.13** 465.81** 513.37 
(1.48) (1.03) (0.88) (0.91) (1.01) 


Note: ***p < .001, **p < .01, *p<.05 


Compared to Malaysian students, the frequency of skipping classes among Singaporean students had an even 
more significant impact on their science exam results across all quantiles after controlling for other variables in the 
model. The influence of absenteeism on exam results was most pronounced at q25 and q50 but less at q90. At q25, 
students who were absent at least once a week scored 135.7 points lower than those who rarely or never skipped 
classes (Table 6). However, absenteeism is rare among Singaporean students compared to their Malaysian counterparts. 

On the other hand, Singaporean students’ perception of science had a more negligible impact on their science 
performance than their Malaysian counterparts, a pattern observed across all quantiles. The impact of students’ 
science perception was most pronounced at q25 and least pronounced at q90 after accounting for other variables 
in the model. 

Educational resources had a much more substantial impact on students’ science performance in Singapore 
than in Malaysia, especially at the lower quantiles. At q10, Singaporean students with access to abundant home 
resources scored 107.6 points higher than those with limited resources, but this difference fell to 50.2 points at q90. 
Moreover, the impact of parents’ education on students’ science performance was more noticeable at the lower 
quantiles than at the higher quantiles. 

Surprisingly, students from schools severely affected by science resource shortages fared better in science 
than those from unaffected or less severely affected schools, controlling for other variables. However, this result 
should be interpreted cautiously, as the score disparity was relatively small, and only a few schools in Singapore 
faced resource shortages. 
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Table 6 


Quantile Regression Coefficients (Standard Errors), Singapore (n = 4,853) 


Variables qi0 q25 q50 q75 q90 


Frequency of absence 


Once a week -123,18"** 135,73" -130.95** 111.99" 89.18" 
(5.66) (3.89) (2.98) (2.64) (3.04) 
Once every 2 weeks -65.55*** -78.02*** 74.37" 52.14" -40.07** 
(4.90) (3.37) (2.58) (2.29) (2.63) 
Once a month 52.64" -53.42** -47.64*** 3771" 31.43" 
(3.35) (2.30) (1.76) (1.56) (1.80) 
Once every 2 months -24.59** -24.97** -18.83*** 12.46*** 3.35" 
(2.49) (1.71) (1.31) (1.16) (1.34) 


Never or almost never (ref) 
Students value science 


Strongly value science 57.52" 65.29*** 61.31 53.99*** 45.09** 
(2.87) (1.98) (1.52) (1.34) (1.54) 

Somewhat value science 27.80"** 32.61** 37.27 36.22*** 28.76*** 
(2.81) (1.93) (1.48) (1.31) (1.51) 


Do not value science (ref) 
Home educational resources 


Many 107.55*** 88.80*** 71.50*** 62.82*** 50.20*** 
(4.48) (3.08) (2.36) (2.10) (2.41) 

Some 73.59*** 66.02*** 52:43" 40.95*** 38.16*** 
(3.71) (2.55) (1.96) (1.74) (1.99) 

Few (ref) 

Parents’ education 

Don’t know 11.08 18.54*** 12.11°* 17.31*** -2.84 
(6.08) (4.18) 3.20) (2.84) (3.26) 

University or higher 68.80*** 56.23*** 44.23*** 43.62*** 23.95*** 
(6.30) (4.33) 3.32) (2.95) (3.38) 

Post-secondary 0.66 6.34 5.41 12.44*** -1.49 
(6.36) (4.37) 3.35) (2.97) (3.41) 

Upper-secondary 19.96** 19.47*** 9.84** 10.68*** -11.19** 
(6.47) (4.44) 3.41) (3.02) (3.47) 

Lower-secondary 4.07 0.98 2.52 15.01*** -4.00 
(7.40) (5.09) 3.90) (3.46) (3.98) 


Some primary or none (ref) 
Science resource shortage 


Not affected 5.09 -7.99"** -7,83"* 3,42" 3.67" 
(3.36) (2.31) (1.77) (1.57) (1.81) 
Affected 13.79% 6.40* 731" 5.23** -2.18 
(3.78) (2.60) (1.99) (1.77) (2.03) 


Affected a lot (ref) 


Constant 377.52" 450.79** 518.88" 563.25 617.52" 
(6.52) (4.48) (3.44) (3.05) (3.50) 


Note: ***p < .001, **p < .01, *p < .05 
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Discussion 


A multitude of factors intricately shape students’ academic performance, particularly in the realm of sci- 
ence education. This study explores the impact of absenteeism and students’ attitudes toward science on the 
academic performance of eighth-grade students in Malaysia and Singapore. These factors comprise a fusion of 
inherent student attributes and their conscientious efforts. Interestingly, despite the contrasting educational 
achievements of the two nations—with Singapore consistently excelling in mathematics and science since 2000, 
while Malaysia faces a substantial performance gap—both absenteeism and science attitudes exhibit remark- 
ably similar influences on exam outcomes in both educational settings. 

Consistent with findings from past studies (Akkus & Cinkir, 2022; Arulampalam et al., 2012; Keppens, 2023; 
Klein et al., 2022; London et al., 2016), which support the faucet theory and school disengagement theory, this 
research distinctly underscores the detrimental impact of absenteeism on students’ performance in science 
across these two countries, which exhibit differing academic standings. This finding highlights the pivotal role of 
factors affecting academic performance, including regular attendance. Quantile regression analysis shows that 
absenteeism had a more pronounced impact at higher than lower quantiles in Malaysia. However, in Singapore, 
its impact was most pronounced at q25 and q50 and least at q90. 

One particularly striking distinction emerges when dissecting class attendance patterns: Malaysian students 
display a threefold higher likelihood of class absenteeism when compared to their Singaporean counterparts. 
This pronounced divergence in absenteeism rates significantly contributes to the observed performance gap 
between these two cohorts of students. Employing standardized analysis, if Malaysian students were to align 
their absenteeism rates with their Singaporean peers, their average science scores would ascend from 460.6 to 
483.8. This result compellingly underscores the potential of addressing absenteeism to narrow the performance 
chasm, ultimately fostering improved academic outcomes in science for Malaysian students. The conspicuous 
dissimilarity in class absenteeism rates between these two groups implies the existence of distinct underlying 
factors influencing their attendance behavior. This study underscores the pressing necessity for a comprehensive 
inquiry into the root causes of this divergence. Given that data on unexcused and sickness-related absenteeism 
are not available from TIMSS, there is a compelling demand for a separate study to elucidate the reasons behind 
class skipping. Such an investigation will pave the way for effective, targeted remedial actions to address the issue. 

This analysis underscores the pivotal role of students’ attitudes toward science in shaping their academic 
performance in both countries, corroborating the findings of numerous other studies and aligning with Eccles’ 
expectancy-value theory (Al-Mutawah & Fateel, 2018; Ali et al., 2015; Chang & Cheng, 2008; Else-Quest et al., 
2013; Juan Andrea et al., 2014; Mao et al., 2021). Notably, among Malaysian and Singaporean students, those 
who strongly valued science (45% and 42%, respectively) achieved significantly higher scores than those who 
held only moderate or negligible appreciation for the subject. In Malaysia, the impact of students’ attitudes 
toward science on academic success was more prominent in lower than higher quantiles. However, there was 
no consistent variation across the score distribution in Singapore. This observation emphasizes acknowledging 
science’s significance and nurturing a positive attitude. 

Educators and policymakers can effectively enhance academic outcomes in science education for Malaysian 
and Singaporean students by cultivating a genuine interest and enthusiasm for science. Fostering students’ rec- 
ognition of the relevance and value of science in their lives can ignite curiosity and passion, ultimately leading 
to increased engagement and superior learning outcomes. Emphasizing the foundational nature of science can 
also enable students to perceive its applicability across various facets of their education and future professions, 
motivating them to excel in their scientific studies. 

This analysis corroborates the findings from many studies (Askin & Oz, 2020; Evans et al., 2010; Ferguson 
et al., 2007; Fergusson et al., 2008; Khan et al., 2019; Li et al., 2020; Li & Qiu, 2018; Liu et al., 2020; Ozkan & Tekeli, 
2021; Sirin, 2005) to emphasize the critical importance of ensuring ample educational resources at home to sup- 
port students’ learning and academic success, particularly in the field of science. Ensuring access to educational 
materials and resources can bridge the achievement gap between students with varying levels of home-based 
resources, both in Malaysia and Singapore. Notably, the availability of internet access also significantly influ- 
ences students’ academic performance (Fernandez-Gutiérrez et al., 2020; Ozkan & Tekeli, 2021; Roman Carrasco 
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& Murillo Torrecilla, 2012; Skryabin et al., 2015; Tan & Hew, 2019), suggesting that disparities in home educational 
resources—such as the number of books and internet access—likely contribute to the observed performance 
gap in science between Malaysian and Singaporean students. 

Parents’ educational level and active involvement in their children’s education emerge as potent factors 
shaping students’ academic achievement, a finding corroborated by prior research (Abu Bakar et al., 2017; 
Davis-Kean, 2005; Murnane et al., 1981). Singaporean students, three times more likely to have parents with a 
university education than their Malaysian counterparts, likely account for part of the differences in their exam 
results. Malaysia's steady progress in educational attainment, with a tertiary enrolment ratio reaching 40% in 
2022, suggests a promising trajectory toward closing the academic performance gap between Malaysian and 
Singaporean students. In both countries, parents’ educational attainment matters more in students’ academic 
success in the lower quantiles than in higher quantiles, as was the case with the availability of home resources. 

The superior academic performance of Singaporean students compared to their Malaysian counterparts 
can be attributed, in part, to differing levels of socio-economic development. Singapore boasts a high human 
development index of .939 (ranked 12th globally) and a per capita income of USD 82,808, whereas Malaysia's 
metrics stand at .803 (ranked 62th globally) and USD 12,471 (United Nations Development Programme, 2022; 
World Bank, 2023). Malaysia's fluctuating policy on teaching mathematics and science in English and the na- 
tional language has presented challenges to learning these subjects, given that most reference materials are 
in English. Investigating Malaysia's underperformance in PISA compared to Korea and Singapore, Perera and 
Asadullah (2019) found that more than half of the achievement gap between the high performers and Malaysia 
still needs to be explained, even after accounting for socio-economic and school-specific factors. This finding 
suggests potential explanations related to policy, institutions, and culture. 

This analysis carries inherent limitations due to secondary data lacking some essential information for 
further examination, such as the reasons for absenteeism and the lack of interest in science subjects. External 
factors are equally crucial, including teachers’ professional development (Tang et al., 2022), teachers’ preparation 
and quality of teaching (Lay & Chandrasegaran, 2018; Shannag et al., 2013), quality and equal science education 
system (Jiang et al., 2023), the learning environment at school and home (Lay & Ng, 2021), and the availability of 
resources. Even broader educational frameworks within a country, including language choices for instruction, 
can significantly impact students’ learning journeys. 

One of the key strengths of this study lies in its potential policy implications. The significant findings related 
to absenteeism, limited home educational resources, and parental education underscore the specific areas where 
educational policymakers in Malaysia can take proactive steps to improve student performance. This study goes 
beyond merely identifying issues; it offers actionable insights that can guide educational planners in addressing 
these shortcomings and enhancing science education outcomes in the country. 

The use of weighted least squares and quantile regression analysis allows this study to examine the net 
effects of various factors on the academic performance of eighth-grade students. By considering other factors 
such as home educational resources and parents’ education, this study also examines the roles of other factors 
affecting the educational faucet’s output. This perspective underscores the importance of effectively tackling 
root causes to improve science achievement. 

This study contributes to the field of science education research, as it allows one to draw insightful com- 
parisons between two countries with contrasting performances in TIMSS rankings. By exploring the impact of 
absenteeism and students’ attitudes toward science in these two contexts, this study offers a valuable contribu- 
tion to understanding science education on a global scale to address educational disparities and improve sci- 
ence education in diverse international contexts. The insights gained from this research can inform educational 
policies and practices in other countries facing similar challenges. Research findings on the factors affecting 
students’academic performance should be disseminated widely to all stakeholders to take appropriate measures 
to ensure that all students acquire the knowledge and skills needed to promote sustainable development, as 
promulgated in the Sustainable Development Goals. 
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Conclusions and Implications 


This analysis has the potential to contribute significantly to advancing educational theories and formulating 
evidence-based policies aimed at improving educational outcomes and reducing disparities, with a specific focus 
on addressing absenteeism and fostering a positive attitude toward science among students. By examining two 
countries with divergent performance levels in science education, this study offers unique insights that can inform 
educational theories and policy initiatives in several ways. 

Firstly, the findings related to absenteeism underscore the critical role of regular attendance in student perfor- 
mance, regardless of the country’s academic standing. The quantile regression analysis highlights how absentee- 
ism impacts students differently across the performance spectrum. This nuanced understanding can inform the 
development of targeted interventions for students at various performance levels. Moreover, the stark contrast in 
class absenteeism rates between Malaysia and Singapore points to country-specific factors influencing attendance 
behavior. Investigating these factors and implementing context-specific strategies can be instrumental in reducing 
absenteeism and improving overall academic performance. 

Secondly, analyzing students’ attitudes toward science offers valuable insights into the significance of nurtur- 
ing a positive perception of the subject. The consistent positive correlation between valuing science and higher 
academic performance holds implications for curriculum design and pedagogical approaches. Educators can con- 
sider incorporating methods that emphasize the relevance and applicability of science in students’ lives, sparking 
their interest and enthusiasm. This approach aligns with the idea of “science capital,’ which suggests that students’ 
attitudes toward science are influenced by their exposure and experiences. Policy initiatives can be designed to 
enhance students’ science capital, thus promoting better learning outcomes. 

Regarding evidence-based policies, the study’s findings suggest that addressing absenteeism and promoting 
positive attitudes toward science should be integral components of education reform efforts. The intervention 
strategies to reduce absenteeism can include counseling and mentoring programs, buddy systems (peer mentor 
programs), and parental involvement initiatives. Moreover, the education system should focus on curriculum en- 
hancements emphasizing science’s practical relevance and real-world applications through internship programs 
and stimulate students’ interest in science subjects. 

Furthermore, this analysis highlights the importance of providing adequate home-based educational resources 
to enhance learning outcomes. Policymakers can explore ways to provide equitable access to resources, such as 
digital learning materials and internet connectivity, especially for students from disadvantaged backgrounds. 
Besides, there must be efforts to support parents in actively engaging with their children’s education. 

This study sheds light on the influence of absenteeism and students’ attitudes toward science on academic 
performance. It offers actionable insights that can enrich educational theories and guide the formulation of evidence- 
based policies. By addressing absenteeism, nurturing a positive attitude toward science, and promoting equitable 
access to resources and parental involvement, educational systems can work toward enhancing outcomes and 
reducing disparities, ultimately fostering a brighter future for students in diverse learning environments. 
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Introduction 
Education for Sustainability 


Sustainability education is both a value and cultural education that 
deals with clarifying the values that guide people in their daily lives, on 
both a personal and social level. Sustainability education also applies 
basic principles in value education; these principles include narrowing 
the gap between a person's beliefs and verbal statements as well as his 
lifestyle and daily behavior, e.g., setting a personal example, making deci- 
sions based on hard facts, as well as considering values (Van Poeck et al., 
2016). Sustainability is generally not defined as a separate field of study, 
but rather, as a transversal field whose content is often integrated into 
other fields of study (Mintz and Tal, 2014). Sustainability education also 
encounters difficulties and challenges regarding implementing transfor- 
mative pedagogies associated with sustainability. These difficulties and 
challenges are reflected in several important cognitive processes, such as 
research, criticism, and reflection, which develop value judgment or critical 
thinking (Sarid & Goldman, 2021). 

Generally, the field of sustainability is currently integrated into cur- 
ricula in various subjects and fields of study, such as agriculture, geography, 
science, and technology, as well as language, in the frameworks of elemen- 
tary education, middle school, and upper school (Mintz and Tal, 2014). 


Storytelling: Characteristics, Components, and Purposes 


In general, stories have the power to immerse audiences in new or 
unusual situations; thus, they provide audiences with new experiences 
through “messengers’, i.e., the characters in the story (Kromka & Goodboy, 
2019; Nwoga, 2000; Hinyard & Kreuter, 2007; Tobias, 2011; Heimes, 2016; 
Woodside, 2010). Storytelling may use an educational or content-oriented 
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Abstract. The use of stories in science 
education can make science more 
interesting and engaging, illustrate 
concepts, and provide opportunities for 
contextual and meaningful learning. Also, 
integrating a scientific story in which the 
sustainability goals are incorporated can 
lead to increasing students’ motivation 

to learn as well as improve and facilitate 
the learning process. Having children ask 
questions allows the educator to learn 
about the child's insights, views, level 

of interest, motivation for learning, and 
knowledge. Therefore, this study focuses 
on the relationship between science 
storytelling, questioning, and sustainability 
among elementary school students. 

It compares the levels of questioning 
among primary students who learned a 
sustainability story in segments versus 
those who learned the story in its entirety. 
A quantitative study was conducted 
involving 120 second graders from the 
Arab sector in Israel. The results revealed 

a significant difference in the two groups’ 
level of questions: the students in the 
experimental group, who learned the story 
in segments, asked higher-level questions 
than those in the control group. The study 
concluded that learning a scientific story 
in segments proved effective in enhancing 
primary students’ questioning ability, and 
that there was a notable preference for 
this segmented storytelling approach over 
traditional whole unit learning methods. 
Keywords: asking questions, science story, 
storytelling, sustainability education, text 
learning 
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approach to stimulate learning processes and conceptual changes (Downs, 2014). When storytelling is used to 
persuade audiences by incorporating a particular message, it can take on a persuasive character and influence 
the audience through stylistic and rhetorical means. In contrast, storytelling aimed at promoting deliberation 
tends to have a more balanced and dialogue-oriented tonality (Arnett et al., 2008), for example, by raising ques- 
tions or presenting opposing viewpoints (Lengauer et al., 2012). 

The advantage of using stories to assess knowledge lies in the fact that the learner is required to take 
the information conveyed to him and formulate it in his own words (Csikar & Stefaniak, 2018). In this context, 
Kromka and Goodboy (2019) emphasized that “good” storytelling performed correctly makes effective use of 
the narrative by directing students to the relevance of the narrative. It creates a clear beginning, body, and end, 
emphasizing students’ evaluation of the content. In addition, it builds connections and finds the missing links; 
it uses a positive emotional tone and ensures that the students understand the context correctly. 

People can learn new facts from stories with some degree of success without prior exposure by using tra- 
ditional teaching methods (Dahlstrom, 2014; Marsh, 2003). Importantly, a digital story helps students improve 
their communication skills, increases their motivation, creates contacts and social connections through a shared 
experience, and serves as a means of increasing their involvement in the learning process (Campbell, 2012; 
Hwang et al., 2016; Kilic, 2014; Morais, 2015). 

Scientific storytelling can be used as a means of conveying complex scientific information in the classroom 
(Csikar & Stefaniak, 2018). A scientific story includes three aspects: (1) the plot’s structure: A typical plot structure 
has six essential components: (i) setting the scene — the introduction or beginning; (ii) a problematic situation - a 
problem that arises, usually of a scientific nature, or science is involved; (iii) a crisis - something must be done by 
the main character due to the same problem; (iv) a critical decision - the main character makes a crucial choice; 
(v) the climax - the story reaches a turning point, when events change rapidly, for better or worse; and (vi) the 
conclusion - the problematic situation is resolved in either a positive or negative way. (2) The outcome of the 
story depends on a critical choice, or the choices made by the main character. (3) Science and the content of the 
essence of science. Depending on its purpose, a scientific story will contain scientific content and/or the nature of 
science. This content should be assimilated from the point of view of the characters and should appear natural 
and necessary so that the story can properly develop (Klassen & Klassen, 2014). 

The use of storytelling that relates to sustainability in the context of education almost inevitably brings 
up a discussion about normativity, i.e., a discussion about the manner of behavior expected of individuals and 
groups ina certain society, behavior that is perceived as acceptable and proper according to the society's values 
(Ehrenfeld, 2008). In the humanistic tradition, education concerns helping learners better relate to the world in 
which they live, successfully deal with the challenges they face, and empower them to change the world for the 
benefit of all (Masschelein, 2004). Storytelling in the context of sustainability contributes to positive changes 
towards sustainability, and it increases the ability of the learners to act with self-determination and persistence. 
It must foster awareness, challenge basic assumptions, clarify values and ideas regarding what kind of sustain- 
ability is desired, and empower individuals and groups to act accordingly (Fischer et al., 2020). 


Reading Comprehension 


It was found that a text that is well organized and that clarifies to the reader the nature of its structure 
in terms of the information it contains, improves the reader's understanding, recall, and ability to apply the 
information presented (Jian, 2019; Liang et al., 2017). A text can be presented either as one complete unit or as 
short segments. These modes of presentation involve two possible procedures for presenting complex teaching 
materials: a conventional procedure and an alternative one. In the conventional procedure, in the first step, a 
complete set of the material is presented, and only then are the elements that comprise it, separately revealed, 
one by one, with the learner controlling the timing of the transition from element to element. The rationale be- 
hind this procedure is that exposing the entire material in the first stage can allow the learner to build a coherent 
context, which helps learning in the second stage, where the learner focuses on each of the elements separately. 
However, the disadvantage of this procedure, especially when there is a high level of interactivity between its 
elements, concerns the overload placed on the learner’s working memory. This could diminish the possibility 
of building a coherent context and could decrease the efficiency of the entire procedure. On the other hand, in 
the alternative procedure, the order is reversed: the elements are first presented separately, one after the other, 
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when the learner has control over the timing of the transition from element to element, and only then is the 
entire material completely presented (Mayer & Chandler, 2001; Mayer et al., 1999). 

Mayer et al. (1999) hypothesized that presenting the elements separately as a first step is more effective 
for complex learning materials. The students in their study watched an animation depicting the formation of 
lightning while listening to a suitable narration. The findings indicated better performance in measures of transfer 
and retrieval in the group of students who watched the animation in small units, compared with the group of 
students who watched the animation in large units. 

Robinson et al. (2003) asked students to study material that included a relatively long text. Some students 
received the original text as one large unit, and the remaining students received the original text in seven small 
text units. In a recall test at the macro level, it was found that presenting large units was more effective than 
presenting small units. The researchers attributed these findings to the design of the conditions of the small 
units in their study. 

In his research, Ayres (2006) examined the effect of a strategy used to separate elements in a solution of 
an algebraic exercise and consequently on students’ learning. The students were requested to focus on solving 
one component of the exercise, and in the next stage, to focus on the next component, and so on. It was found 
that in solving the exercise the strategy of separating the elements reduced the level of cognitive load in both 
groups of students. 


Asking Questions 


In general, having students ask questions is an essential cognitive strategy, since the act of composing 
questions focuses students’ attention on the content and the main ideas as well as determines whether the 
content is properly understood (Yu et al., 2015; Su & Chen, 2018). Garcia and colleagues (Garcia et al., 2014) 
added that asking questions is a cognitive strategy that fosters understanding; as a learning strategy, it is in- 
volved in creating meaningful learning. This strategy was found in previous studies to have positive effects on 
student performance in various aspects, such as understanding, motivation to learn, positive attitudes towards 
the subject studied, more diverse and flexible thinking, problem-solving skills, and developing a cognitive and 
metacognitive strategy (Chin & Brown, 2002; Chin & Osborne, 2008; Song et al., 2017; Yu et al., 2015; Yu & Liu, 
2008; Yu et al., 2013). According to the classification of Pizzini and Shepardson (1991), students’ questions can 
be divided into three types according to the cognitive level of the questions: input, processing, and output. 
Input-level questions require students to remember information from sensory data. In contrast, processing-level 
questions require them to draw connections between the data, and output-level questions require them to go 
beyond the data to hypothesize, generate, and evaluate information. 

The strategy or skill of asking questions in science has many advantages, as reflected in the goals of using 
this strategy. These goals include developing curiosity and stimulation towards further learning, developing cre- 
ative thinking, providing all students with an opportunity to express their personal points of view and interests, 
focused thinking, experiencing a critical or important stage in the scientific research process, developing the 
ability to link several factors using prior knowledge, developing the ability to identify a problem, legitimizing 
uncertainty in learning and science, and creating an opening for a research/learning process (Chin & Osborne, 
2008). Despite the many contributions and benefits of having students ask questions, as described above, ac- 
cumulating evidence indicates that most of the questions in the classroom are asked by the teachers, and onlya 
minority are asked by the students (Eshach et al., 2014; Kaya, 2014; Su & Chen, 2018). Furthermore, the questions 
that students ask are of a relatively low level (Garcia et al., 2014). A variety of reasons and factors can explain 
the small number of questions asked by students in class and their low level, for example, the didactic teaching 
style of the teacher, a competitive and unfriendly atmosphere in the classroom (Jung et al., 2016), and limited 
teaching time (Chin & Osborne, 2008). 

Blonder et al. (2008), who examined students’ skills of asking questions during a chemistry laboratory 
class, classified their questions into three levels: Level 1- questions regarding the topic under discussion; Level 
2 - questions regarding scientific equipment and scientific methods; and level 3 - questions concerning the 
phenomenon in real life. They found that the highest and most complex level of the students’ questions is level 
3 because it requires them to understand the subject being studied, the tools and methods of measurement, 
and above all, the ability to transfer this knowledge to real-life situations. However, they found that students 
rarely ask level-3 questions. 
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In science classrooms, asking questions is the most crucial aspect of inquiry learning (Tan & Seah, 2011). 
Learning should be designed to enable students to build their own knowledge (Zion & Sadeh, 2007). In inquiry 
learning, instead of teacher-guided materials, students’ curiosity drives the lessons. In this learning framework, 
students are free to ask questions, gather information, and analyze it to answer the questions (Jorgenson et al., 
2004). Because students are actively involved in these processes, they retain more of their learning for a longer 
time and therefore are more motivated to learn (Pedrosa-de-Jesus & Watts, 2014). 

Asking or creating questions is a metacognitive process because it focuses students’ attention on the content 
and promotes deeper thinking. The nature of students’ questions indicates their understanding of the course 
material (Chin & Osborne, 2008), as well as their misconceptions about the subject (Pedrosa-de-Jesus & Watts, 
2014). Asking questions may even help teachers determine the direction of the lesson and see the teaching 
materials from different perspectives (Chin & Brown, 2002). A certain period (sometimes years) is considered 
critical in terms of developing the ability to create questions in general, and at higher levels (Kaya & Temiz, 2018). 

Considering the importance of students asking questions, the current study will present a possible 
framework linking it to the important teaching practice of telling stories, which can help elevate the level of 
students’ questions. It will relate to the issue of sustainability, a cultural value issue that is topical and of great 
importance nowadays. More specifically, the study will describe the positive effect of storytelling in relation to 
sustainability and how it contributes to primary school students by elevating their level of asking questions on 
both the theoretical and applied levels. At the theoretical level, the research expands the existing knowledge 
regarding science students’ ability to ask questions. This topic has recently gained increased interest in the 
scientific-educational literature. At the applied level, and based on the current research, the development and 
implementation of the scientific topic “the sustainability story in short segments” encourages and promotes the 
creation of questions among the students and provides them with practical strategies and tools to promote the 
skill of asking high-level questions. 


Problem 


The research problem addressed in this study revolves around the skill of asking questions among students 
in the context of learning a narrative text about sustainability. Specifically, the study examined whether the 
mode of presenting the narrative text (in short segments versus in its entirety) affects young students’ level of 
questioning. The central goal was to determine how storytelling, when utilized as a teaching practice in sustain- 
ability education, can influence the depth and complexity of students’ questions. 


Aim and the Research Question 


This study compared two modes of learning a story: one where the narrative is presented in short segments 
and another where it is presented in its entirety. The main interest lies in researching how these two modes affect 
the level of young students’ questioning in the context of sustainability education. The research question is: Is 
there a difference between the level of questions asked by students who studied a story about sustainability in 
short segments and those who studied it in its entirety? 


Hypothesis 
The hypothesis is that there is a difference in the level of questions asked by students who studied a story 
about sustainability in short segments and those who studied it in its entirety: the level of questions of students 


who studied the story in short segments would be higher than the level of questions of those students who 
studied it in its entirety. 
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Research Methodology 
General Background 


The research hypothesis was tested using the quantitative approach due to its suitability for the present 
study: an objective examination (as much as possible) of the correlations between variables without the re- 
searcher’s personal interpretation or conclusion. In addition, the quantitative approach (a quasi-experimental 
design, which makes it possible to carry out experiments under field conditions) allows a high level of research 
generalization and maintains the principle of repeatability, and its reliability is relatively high (compared with 
the reliability of qualitative research) (Creswell, 2014). 

Each of the students in the control group received a complete story on sustainability from the teacher; they 
studied it, and finally answered the questions that appeared at the end of the story for two consecutive lessons 
(90 minutes). In contrast, each student in the experimental group received the same story from the teacher; 
however, the story was divided into short sections. After the students read a certain passage, they answered 
the questions that appeared at the end in two consecutive lessons (90 minutes). The students in both groups 
wrote their answers regarding the text. Next, the students were asked to write questions that came to them 
after they had read the story. 

The procedure of the research was carried out during science lessons for second grade students in the 
2022/2023 academic year; the students were taught by two science teachers in each of the schools that partici- 
pated in the study. Note that the teachers did not intervene during the study, apart from minimal interventions 
of clarifications and explanations to the students, if necessary. The full texts were collected by the teachers at 
the end of the lesson. 


Sample 


The study involved a sample of 120 students from four second-grade classes in two primary schools 
located in the Arab sector in northern Israel. Each school had two second-grade classes, and the participants 
were randomly divided into two groups: an experimental group (N=30) and a control group (N=30); each group 
comprised students from one grade. 

Several factors were considered: 

1. Representation: Given the localized nature of the study within the Arab sector in northern Israel, 
having 120 students from four different second-grade classes from two primary schools helps in 
obtaining a fair representation of the population in this specific geographic and cultural setting. 

2. Randomized Group Allocation: The division of participants into experimental and control groups, 
each with 30 students, allows making a balanced comparison between the groups. This balanced 
design enhances the statistical power of any comparisons made, providing a clearer understanding 
of any effects observed. 

3. Homogeneity: The relative homogeneity of the sample with respect to key variables such as gender, 
learning disabilities, academic achievements, and socioeconomic status, helps to control potential 
confounding variables. This control increases the likelihood that any observed differences are due 
to manipulation in the study rather than to extraneous variables. 

4. Convenience Sampling: Although convenience sampling was used due to the accessibility of the 
students to the researcher, it was thought that the size of 120 students might help in overcoming 
some of the biases associated with this non-random sampling method by providing a larger pool of 
participants from which to draw conclusions. 

5. Statistical Power and the Effect Size: The sample size of 120 can offer a good balance between 
achieving sufficient statistical power to detect effects (if they exist) while remaining manageable in 
terms of resources and logistics. 

6. Practicality: The sample size is also justified on practical grounds, since it aligns with the resources, 
access, and time available to the researcher. 

In sum, the sample size of 120 students seems to be a well-considered choice given the localized setting, the 
study design, and the practical constraints faced by the researcher. 
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In computational terms, the formula for calculating the required sample size for a proportion in a large popula- 
tion, when the population’s proportion is known or assumed, is given by: 
n=(Z/E)-p(1-p) 
where: 
e Zis the z-score (which corresponds to the desired confidence level, 1.96 for 95% confidence). 
e Eis the margin of error= .05 
e pis the estimated proportion= .91 


n= (1.96/.05)?*0.085(1-0.085) =120 


The sample of students was relatively homogeneous with respect to variables such as gender, the number of 
children with learning disabilities, academic achievements, and socioeconomic status. For the sake of good order 
and orderly ethics and before the texts (of both types) were transferred to all classes, the study was presented 
and all participants were asked to give their consent to participate, it was made clear that the worksheets would 
be anonymous, and the data would be used for research purposes only. Also, and for the sake of professional 
ethics, after the end of the research, the control group that studied the text in its entirety, the teacher was asked 
to choose an environmental science story and teach the text in a fragmented manner. 


Instrument and Procedures 


The first story, “Rose in the Dark’, is a symbolic story that demonstrates to the student the importance of 
taking care of the Earth in which we live, and not polluting the air so that adequate light reaches us. The second 
story, “The Polluted Lake’, is about a group of animals that live near a lake. A real disaster occurs because of 
sewage water pollution. The animals decide to deal with the problem of environmental pollution to preserve 
their natural environment. 

The two stories were compiled by Hugerat (2007). The stories are intended to increase students’ awareness 
of sustainability. Importantly, they even strengthen three additional aspects for the students: the scientific- 
environmental aspect, the social aspect, and the educational aspect. The stories are accompanied by questions 
that were also composed by Hugerat (2007). The stories are in Arabic, the native language of the research sample. 
The two stories, including the questions about their content, in order to evaluate the students’ understanding, 
were presented in two ways: as a complete story, and as a story composed of short segments that have a gradual 
connection between them. The students in the two classes were randomly divided into two groups: one group 
served as a control group (that learned the subject of sustainability through a complete story), and the second 
group was an experimental group (that learned the subject of sustainability through a story that appeared 
in short segments). Each group studied for three lessons; each lesson lasted 45 minutes. The described study 
procedure was carried out by two science teachers; as mentioned, the teachers did not intervene during the 
study, except for minimal interventions of clarifications and explanations to the students, if necessary. The full 
texts were collected by the teachers at the end of the lesson. 


The research was carried out in the following stages: 

1. Each story was presented in two ways: as a complete story and as a story consisting of short segments 
that maintain a gradual connection between them. The questions in the complete story appeared 
at the end, whereas the questions in the story in segments appeared after each segment, accord- 
ing to its content. The two ways in which the stories were presented in the study (described in the 
previous segment) were examined by two experts who have a PhD in science teaching and who 
have a deep knowledge of the subject. They were asked to refer to the following: the clarity of the 
stories and questions, how well the stories and questions were adapted to the level of the research 
sample, how well the short segments were adapted to the purposes of the research, as well as the 
length of the questions. Following their comments, the questions were changed accordingly. Then, 
two Arabic language researchers with PhDs were asked to comment on the interpretation of the 
sentence’s meaning, wording, and clarity. The comparison at the end of the process showed that 
there was “almost” full agreement of the assessments of the two lecturers. The comments made to 
the research editor focused on grammatical and clarity issues. 
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2. The students in the two classes were randomly divided into two groups: one group served as a con- 
trol group (they learned about sustainability through a complete story), and the other group was an 
experimental group (they learned about sustainability through a story that appeared in short seg- 
ments). In the pre-intervention phase, the basic question levels of the students in the control group 
and the experimental group were tested. In this test, the students read a science story presented in 
its entirety. Similarly, the students’ level of understanding of the story they had read was also tested. 
The students answered questions about the content of the story. In the intervention phase, each 
student in the control group read a complete story about sustainability, studied it, and then answered 
the questions that appeared at the end of the story. In contrast, each student in the experimental 
group read the same story, which was divided into short segments. After the students read a certain 
passage, they answered in writing the questions that appeared at the end. Next, the students were 
asked to write down any questions that they had after they had read the story. 

3. Inanother stage after the intervention, the level of the questions and the level of understanding of 
the two groups of students were determined again, after the intervention. 

4. The data were analyzed regarding the level of the questions that the students raised about the 
content of the story they had read. In addition, the data obtained from the students’ answers to the 
questions were analyzed regarding their understanding of the content in the stories. 


Data Analysis 


To examine the research hypothesis, the data collected on the level of the students’ questions (the types 
of questions) in both groups were evaluated and coded according to Bloom's taxonomy (Bloom, 1956). Bloom 
described six levels of questions (1-knowledge, 2-understanding, 3-application, 4-analysis, 5-synthesis (fusion), 
and 6-evaluation), according to the level of thinking that they activate in the questionnaire (from the lowest to 
the highest level). 

In the next step, to examine the research hypothesis, a t-test for independent samples was conducted to de- 
termine whether there was a significant difference in the level of questions asked by students in the experimental 
group (learning the story in short segments) and students in the control group (learning the story in its entirety). 

In addition, the students’ answers to the questions that appeared at the end of the story (the control group), 
or after reading each passage (the experimental group) were analyzed. The data collected on the answers of 
the students in both groups were evaluated and coded according to seven criteria as follows: (1) the sentence 
structure — the creation of a sentence that makes sense and has internal coherence, (2) causality — the use of 
inference and reasoning while using an explicit expression that affects relationships, (3) direct reference of the 
answer to the question - understanding the question and organization in writing the answer; (4) coherency: 
the fluency of words - the total number of words used by the writer; (5) the fluency of ideas - the total number 
of ideas expressed by the writer in each answer; (6) knowledge - the level of knowledge shown by the student 
on the topic being studied; and (7) originality - reference to answers that indicate unconventional thinking, 
where the rating for each criterion ranges from 1 to 6, 1- the lowest rating, up to 6 - the highest rating (Calpon 
& Eshel, 1999). 

In the next step, to evaluate the students’ answers, an additional t-test for independent samples was 
conducted to determine whether there was a significant difference in the students’ level of understanding and 
their knowledge of the subject being studied (the content of the story) between students in the experimental 
group) and in the control group. 


Research Results 
Descriptive Results 
Before testing the research hypothesis, the descriptive indices (e.g., minimum, and maximum values, mean 


and standard deviation) of the research variable were examined in the two groups, in both measurements (pre-test 
and post-test). The results are shown in Table 1. 
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Descriptive Indices of the Research Variable - The Level of the Students’ Questions 


Variable Min Max M SD 
Control Group- 
First measurement 1 2 1.03 0.57 
Second measurement 1 2 1.15 0.95 
Experimental Group- 
First measurement 1 3 1.4 0.8 
Second measurement 1 4 3.10 1.20 


As shown in Table 1, the averages in the first measurement of the general question level in the experimental 
group and the control group were lower than in the center of the scale. In the second measurement, the averages 
of the general question level in the experimental group were in the center of the scale (3), and in the control group, 
they were lower than the center of the scale. Thus, the questions asked by the students in the control group and 
the experimental group (respectively) were at a low level in the first measurement, and at a low-to-medium level 
in the second measurement. 


Inferential Results 


As mentioned, the research hypothesis was that the level of questions from students who studied the story in 
short segments (the experimental group) would be higher than the level of questions from students who studied the 
story in its entirety (the control group). To test the research hypothesis, a two-way mixed design ANOVA was used. 
The between-subjects independent variable was the study group, and the within-subjects independent variable 
was the measurement time. The dependent variable was the measured values. Two such analyses were performed: 
one for the variable of the level of questions asked by the students, and one for the variable of the students’ level 
of understanding. First, the variable level of students’ questions was tested. The results are presented in Tables 2-4. 


Table 2 
Means and Standard Deviations of the Level of the Student's Questions, Divided by the Measurement Time 


Experimental Group Control Group 
Levels of Student’s Questions 
M SD M SD 
First measurement 1.40 0.80 1.03 0.57 
Second measurement 3.10 1.20 1.15 0.95 


Table 3 
Results of the Analysis of Variance for the Level of the Students’ Questions - Variables within Subjects 


The Source of Variation Ss df Ms F p 
The Measurement Time 3.41 1 3.41 224.30 < .001 
Interaction - Measurement Time X Study Group 3.32 1 3.32 212.52 < .001 
The Error Factor (Time of Measurement) 0.53 116 0.08 
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Table 4 


Results of the Analysis of Variance for the Level of the Students’ Question Variables between Subjects 


The Source of Variation SS df Ms F p 
The Research Group 5.73 1 5.73 36.39 <.001 
The Factor of Error (Study Group) 7.06 116 0.28 


As shown in Table 3, in accordance with the research hypothesis, a significant interaction was found between 
the study group and the measurement time (F (1,116) = 212.52, p < .001). In addition, significant main effects were 
found for the measurement time (F (1,116) = 224.30, p < .001) and the study group (F (1,116) = 36.39, p < .001). 

To locate the source of the interaction, a Bonferroni post-hoc test was used. In accordance with the research 
hypothesis, it was found that in the experimental group, the level of questions asked by the students in the sec- 
ond measurement (M = 3.10, SD= 1.20) was significantly higher (p < .001) than in the first measurement (M = 1.40, 
SD = 0.80). It was also found that in the control group, the level of questions asked by the students in the second 
measurement (M = 1.15, SD = 0.95) did not significantly differ (p = .735) from the first measurement (M = 1.03, 
SD = 0.57). Therefore, the research hypothesis was confirmed. 

In addition, the variable level of students’ understanding was tested. The results are presented in Tables 5-7. 


Table 5 
Means and Standard Deviations of the Students’ Level of Understanding, Divided by the Measurement Time 


Experimental Group Control Group 
Students’ Level of Understanding 
SD M SD M 
First Measurement 0.66 1.37 0.81 1.05 
Second Measurement 1.40 4.20 1.84 1.55 


Table 6 
Results of the Analysis of Variance for the Students’ Level of Understanding - Variables within Subjects 


The source of variation SS df Ms F p 
The Measurement Time 1.76 1 1.76 158.30 < .001 
Interaction - Measurement Time X Study Group 1.63 1 1.63 189.46 < .001 
The Error Factor (Time of Measurement) 0.38 116 0.09 


Table 7 
Results of the Analysis of Variance for the Students’ Level of Understanding - Variables Between Subjects 


The source of variation SS df Ms F p 
The Research Group 8.65 1 8.29 27.09 < .001 
The Factor of Error (Study Group) 12.07 116 0.63 


As shown in Table 6, in accordance with the research hypothesis, a significant interaction was found between 
the study group and the measurement time (F (1,116) = 189.46, p < .001). In addition, significant main effects were 
found for the measurement time (F (1,116) = 158.30, p < .001) and the study group (F(1,116) = 27.09, p < .001). 

To locate the source of the interaction, a Bonferroni post-hoc test was used. In accordance with the re- 
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search hypothesis, it was found that in the experimental group, the students’ level of understanding in the 
second measurement (M = 4.20, SD = 1.40) was significantly higher (p < .001) than in the first measurement 
(M = 1.37, SD = 0.66). It was also found that in the control group, the level of the students’ understanding in 
the second measurement (M = 1.55, SD = 1.84) did not significantly differ (p = .441) from the first measurement 
(M = 1.05, SD = 0.81). From these findings, apparently, students who learned about sustainability in short seg- 
ments of a story improved their level of understanding in comparison with those who learned it in its entirety. 


Discussion 


Considering the importance of the skill of asking and answering questions by science students, especially 
among students at the beginning of their schooling, and since this skill is hardly reflected in science classes, this 
study tried to examine a teaching method that may elevate the level of students’ questions. Specifically, it exam- 
ined how the learning mode of a science story on sustainability, presented in short segments, affected the level 
of questions asked by students. 

As hypothesized, it was found that students who studied the story presented in short segments asked ques- 
tions at a higher level than students who studied the story presented in its entirety: the average level of questions 
of students who studied the story on sustainability continuously in short segments (M = 3.10) was higher than the 
average level of questions of students who studied it as one unit (M = 1.15) (Table 2). 

The research results, which generally indicated the usefulness of a science story presented in short segments, 
compared to a whole unit, are consistent with previous empirical findings that showed that learning a text in small 
units yields better results than learning it in one large unit (Mayer & Chandler, 2001; Mayer et al., 1999; Minaee et 
al., 2021; Yang et al., 2020). 

A possible explanation that supports the research findings is based on the theory of cognitive load in learn- 
ing. Cognitive overload occurs when the demands placed on the learner by the task exceed the capacity of the 
learner's attentional resources (Sweller, 1994; 2011). In this situation, both learning and the learner's performance 
are adversely affected, as found in Ayres’ research (Ayres, 2006). Cognitive load theory assumes that there are three 
types of cognitive load: internal load, external load, and relevant load (Moos & Pitton, 2014; Tricot et al., 2020; Zhou 
& Lamberton, 2021). Internal load refers to the degree of complexity of the subject being studied; it is defined ac- 
cording to the number of elements that the learner’s working memory must process at once, i.e., the interactivity 
of the elements. The complexity of the task also depends on the expertise of the learner. An assignment that is 
considered simple and with few elements for an expert learner is a complex assignment with many elements for 
a beginning learner. External load is created in a teaching environment that requires the learner to engage in ac- 
tivities that are not directly related to reaching a consensus or developing automaticity. Since working memory is 
limited, engaging in these activities places a load that is not conducive to learning. When the internal and external 
loads exceed the limits of the learner's working memory, learning is impaired. A relevant load can be defined as the 
mental effort required to learn content. This load is created by cognitive processes relevant to learning (processing, 
construction, and automation of schemes), and it is effective for learning. 

Therefore, it can be assumed that the students in the control group, who learned the story as a whole unit, 
experienced cognitive overload when reading the story in its entirety, answering all the questions at the end, 
and especially when they were asked to ask questions themselves. This overload consisted of three types: (1) an 
internal load that resulted from the complex subject of sustainability that was learned through the story, (2) an 
external load resulting from the task of creating questions on their own about the content of the story, and (3) a 
relevant load that probably resulted from the great mental effort they had to make in processing information ona 
new topic (sustainability), which was presented to them as one long narrative text. In contrast, the students in the 
experimental group, who studied the text in short segments, experienced a lower level of cognitive load, because 
the text was presented to them in short segments. Consequently, they were able to ask questions at a higher level. 

Support for the above explanation, based on the theory of cognitive load in learning, was obtained through 
the research findings that indicated that students who studied a story continuously through short segments had a 
better understanding of it than did students who studied the story as a whole unit (Tables 5-7). Therefore, it can be 
concluded that the students in the experimental group learned more than their counterparts in the control group 
due to the lower level of cognitive load in the experimental group and that this led to a higher level of questions 
that they asked compared with their peers in the control group. 
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This finding perhaps indicates a connection between understanding educational content and the level of 
questions that can be asked about it. This assumption is supported by the claim in the literature that asking ques- 
tions by students reflects their level of understanding the subject being studied (Bergey et al., 2022; Chin & Brown, 
2002; Chin & Osborne, 2008; Maplethorpe et al., 2022; Song et al.., 2017; Su & Chen, 2018; Touissi et al., 2021; Yu et 
al., 2013, 2015; Yu & Liu, 2008). 

Finally, the research results can be explained through the cognitive benefits and contributions attributed to 
the presentation of an educational text in short segments. Dividing the text into short segments, accompanied by 
questions to which the reader is asked to answer, helps the reader to focus on the task assigned to him and directs 
his attention to the pieces of information in the text that can help complete it. Therefore, it can be assumed that a 
short segment with questions about its content at the end promotes cognitive processes that ultimately make it 
possible for students to ask questions at a higher level. An important limitation is that in the statistical analysis, the 
research did not examine how the students’ understanding of the story segments they had read affected the level 
of the questions they asked. It is reasonable to assume that students who understood better and more accurately 
the content they read could ask questions at a higher level than students who did not sufficiently understand 
what they had read. 


Conclusions and Implications 


The research results yield three significant conclusions: First, they demonstrate that the level of questions 
posed by young learners concerning sustainability shows a tendency towards being low to moderate, indicating 
a predominantly superficial cognitive processing approach. Second, the study provides supporting evidence for 
the efficacy of adopting a segmented approach in learning scientific narratives. It was shown that this approach 
effectively elevates the level of questioning among primary school students. Third, the results indicate a clear 
preference among primary school students for the segmented mode of learning when studying science subjects, 
as opposed to traditional whole-unit learning. 

The study reveals that regardless of the learning method employed (i.e., short segments versus complete 
narrative), the overall level of questions regarding sustainability posed by primary level students in the Arab sector 
remains within the low to moderate range. However, students who learned the story in segmented form exhibited 
higher-level questioning compared with their counterparts who learned the story as a unified unit. 

Consequently, the study advocates integrating scientific narratives tailored for segmented learning in primary 
school science curricula. Thus, it is imperative for primary science educators to incorporate these research insights 
into their professional development strategies. By employing practical approaches and utilizing resources according 
to the study’s findings, educators can effectively break up scientific narratives into smaller, digestible segments, 
fostering curiosity among students and can stimulate the students to ask thought-provoking questions. 

Finally, the research findings can be explained through the cognitive benefits and contributions attributed 
to presenting an educational text in short segments. Dividing the text into short sections, accompanied by ques- 
tions to which the reader is asked to answer, helps the reader to focus on the task assigned to him and directs his 
attention to the pieces of information in the text that can help complete the task. Thus, it can be assumed that a 
short section with questions about its content at the end promotes cognitive processes that ultimately make it 
possible to ask questions at a higher level. 
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MODERATING ROLE OF 
SCIENCE SELF-CONCEPT IN 
ELICITING STATE CURIOSITY 
WHEN CONFRONTING A 
VIOLATION OUTCOME 


Jihoon Kang 


Introduction 

One of the main goals of science education is to change students’ 
misconceptions into scientific concepts. One of the proposed approaches 
to effectively guide students’ misconceptions into scientific concepts is the 
strategy of inducing cognitive conflict (Lee et al., 2003; Niaz, 1995). While 
presenting situations of incongruity alone does not guarantee the occur- 
rence of cognitive conflict, it is imperative to have incongruent situations for 
cognitive conflict to arise (Kwon et al., 2003). Therefore, in science education, 
emphasizing the presentation of phenomena or data that are inconsistent 
with students’ preconceptions has been emphasized as a means to rectify 
their misconceptions (Lee & Kwon, 2001; Lee et al., 2003; Stahl & Feigenson, 
2017). 

Students tend to become more attentive when they encounter novel 
and intriguing stimuli that do not align with their expectations during the 
learning process, and curiosity moderates this process (Day, 1982; Gruber 
& Ranganath, 2019; Loewenstein, 1994). Students who confront phenom- 
ena or results that do not align with their expectations become aware of a 
knowledge gap, which is a discrepancy between what they currently know 
and what they want to know, and their curiosity is then aroused to resolve 
this gap (Berlyne, 1960; Grossnickle, 2016; Loewenstein, 1994; Shin & Kim, 
2019). Curiosity in learning has a positive impact on attention, memory, and 
retention, enabling effective learning, and is also helpful in fostering scientific 
thinking and science literacy (Bathgate et al., 2014; Gruber & Ranganath, 2019; 
Gruber et al., 2014; Kang et al., 2009; Marvin & Shohamy, 2016; Murphy et 
al., 2021). Given that curiosity has a significant influence on academic per- 
formance and achievement in science learning (Authors, 2021; Bathgate et 
al., 2014; Weible & Zimmerman, 2016), it is necessary to induce a high level 
of curiosity for effective learning. 

Given the significant impact of students’ curiosity on academic perfor- 
mance, it is crucial to identify their curiosity levels in the learning situation 
and respond accordingly. At this point, curiosity refers to an individual's 
emotional state whose level can relatively easily fluctuate depending on 
the situation, not as a personal innate trait that does not change easily 
(Loewenstein, 1994). State curiosity, which denotes temporary curiosity in 
specific learning situations, can be altered in its intensity through appropriate 
educational interventions. Therefore, it should be given significant attention 
in the research of science education. 
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Abstract. A higher science self-concept 
may be required to stimulate state curiosity 
in incongruent situations, but there is 
limited research on the connection between 
science self-concept and state curiosity. The 
purpose of this study is to examine whether 
science self-concept moderates the process 
of arousing students’ state curiosity when 
they encounter results that violate their 
expectations in science learning. To achieve 
this aim, 410 fifth- and sixth-grade primary 
school students (194 female) were asked 

to solve science questions, and students 
who faced results that deviated from their 
expectations were classified as the violation 
outcome group (experimental group), 
while those who faced results consistent 
with their expectations were classified as 
the expected outcome group (comparison 
group). Then, the moderating effect of 
science self-concept on the relationship 
between science curiosity and state 
curiosity arousal in these two groups was 
verified using PROCESS macro. Results 
showed that science curiosity significantly 
predicted state curiosity in both groups. 
Notably, science self-concept hada 
significant moderating effect on the process 
of arousing state curiosity in the violation 
outcome group, where higher science 
self-concept led to a greater increase in 
state curiosity after confirming the correct 
answer. However, in the expected outcome 
group, the moderating effect of science 
self-concept was not significant. The 
implications of these findings for science 
education and potential directions for 
future research are discussed. 
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To stimulate curiosity, which is a desire for knowledge that an individual does not currently know, they should 
have the belief and confidence that they can bridge specific knowledge gaps they currently perceive (Deci, 1975; 
Pekrun, 2019; Peterson & Cohen, 2019). Because moderate levels of perceived knowledge gaps can stimulate high 
levels of state curiosity (Kang & Kim, 2020; Litman et al., 2005; Loewenstein, 1994), it becomes challenging to arouse 
state curiosity when individuals face questions perceived as too difficult or when they lack the belief that the gaps 
can be resolved. Emotions experienced during learning are often influenced by the perceived controllability of the 
task (Pekrun et al., 2007). If a task is perceived as controllable, state curiosity may be more likely to be instigated. 
For example, when a student feels capable of dealing with specific tasks, they are more motivated to learn new 
knowledge and information to solve the task, resulting in a heightened state curiosity. Conversely, when a stu- 
dent holds low expectancy beliefs regarding their ability to succeed in the task, curiosity may remain low or not 
be stimulated, as they may be less inclined to challenge themselves to acquire new knowledge and information. 
Therefore, a relatively high level of self-concept may be required to elicit state curiosity in discrepant situations. 
Self-concept pertains to an individual’s subjective judgments, beliefs, and perceptions about themselves, forming 
a multifaceted and hierarchical construct (Baumeister, 1999; Shavelson et al., 1976). Academic self-concept (e.g., 
science self-concept), a subcomponent of self-concept, pertains to domain-specific perceptions about oneself in 
academic-related achievement situations. Therefore, students with a high academic self-concept in a specific field 
exhibit confidence and a sense of superiority in their competence in that particular subject area (Bong & Skaalvik, 
2003; Ferla et al., 2009). Hence, students with a high self-concept in science may be more curious due to their belief 
that they can comprehend and figure out scientific phenomena, even when those phenomena challenge their 
preconceptions or expectations. In contrast, students with a low science self-concept lack confidence or belief in 
resolving knowledge gaps when they encounter unacquainted scientific knowledge or information. Consequently, 
their curiosity may be low even when they recognize knowledge gaps. 

It is challenging to find empirical studies that examine whether state curiosity levels vary depending on the level 
of the student's science self-concept when state curiosity is aroused by encountering phenomena that contradict 
their expectations in science learning. If research is conducted on the level of state curiosity aroused depending on 
the level of self-concept, it can establish a theoretical foundation for understanding the relationship between self- 
concept and curiosity, which is known to strongly predict academic achievement. This could offer valuable insights 
to researchers and practitioners in science education, providing them with significant educational implications. 

Accordingly, in this study, an attempt is made to ascertain whether science self-concept plays a moderating 
role in the process of arousing students’ state curiosity when they are confronted with scientific phenomena that 
deviate from their expectations. Through this, implications for science education are aimed to be provided by 
examining the role of the science self-concept in arousing state curiosity. 


Research Question 


Does students’ science self-concept moderate the relationship between their science curiosity and the arousal 
of state curiosity when they encounter results that violate their expectations? 


Research Methodology 
General Background 


This research was attempted within the natural setting of a classroom, where education normally transpires, 
with an effort to minimize artificial intervention or manipulation. As a result, this research was conducted in the 
research participants’ classroom in the context of a general science class. This approach, which observes and 
measures participant behavior as it naturally unfolds in regular learning environments, guarantees substantial 
ecological validity (Jirout et al., 2018). 

The current study was conducted with primary school students in a major city in South Korea. The students 
came from families with a socio-economic status deemed as upper middle class. Moreover, the students exhibited 
an above-average level of academic achievement and displayed a notable interest in their schoolwork. 

This study is based on quantitative methods. The hypothesis in this study posits that the relationship between 
science curiosity and state curiosity would be moderated by science self-concept when students are faced with 
outcomes deviating from their expectations (see Figure 1). Accordingly, cases where participants encountered 
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results that contradicted their expectations were classified as the “violation outcome” group, whereas those that 
did not were classified as the “expected outcome” group. In the conceptual model (Figure 1) designed to test the 
hypothesis, the independent variable was designated as science curiosity, the dependent variable as state curiosity, 
and the moderating variable as science self-concept. In this study, curiosity was analyzed by dividing it into two 
components: science curiosity, which pertains to long-term aspects, and state curiosity, which pertains to short- 
term aspects (see Ainley, 2019; Grossnickle, 2016). Science curiosity is defined as a relatively stable tendency for 
individuals to experience curiosity regarding natural phenomena within the realm of scientific knowledge (Kang et 
al., 2020; Spektor-Levy et al., 2013; Weible & Zimmerman, 2016). This curiosity is distinctive from epistemic curiosity 
about general knowledge. In contrast, state curiosity is considered as a transient emotional state, characterized by 
amomentary curiosity evoked in specific situations (Loewenstein, 1994). Grade, gender, question type, interest in 
the content of questions, need for cognition, and confidence in one’s answer, all of which are plausible to affect 
state curiosity, were held as covariates. 


Figure 1 
Conceptual Model of the Relationship between Science Curiosity and State Curiosity, and the Moderating Role of Science 


Self-Concept 
| Science self-concept 
! State curiosity 


Control variable 


* Grade * Gender * Question type 
* Interest * Need for cognition * Confidence 
Participants 


This study was conducted with a total of 410 primary school students (194 female) in the 5th and 6th grades. 
Given that each student responded to two questions (designated questions A and B), the maximum possible number 
of study cases is 820. Among them, invalid cases, such as those where participants indicated a lack of comprehen- 
sion of the question, instances displaying insincere responses (e.g., illegible handwriting), cases marked by unclear 
labeling, and those containing at least one missing response, were excluded from the analyses. In addition, students 
who had low confidence in their answers (the level of confidence was less than 3) and did not write down reasons 
for their answers were excluded because it was judged that their preconceptions regarding the question were not 
firmly established. As a consequence, a final analysis was conducted on 547 cases. With this sample, there were 
293 cases in the violation outcome group (132 female), and 242 cases in the expected outcome group (98 female). 


Procedures 


The procedure of assessments and treatments in this study is illustrated in Figure 2. First, students were re- 
quested to rate their level of science curiosity, interest in the subject matter covered by science questions A and B, 
need for cognition, and science self-concept. Subsequently, all students were presented with either science question 
A or question B and were asked to try to answer them. Question A pertains to thermal conductivity according to 
materials and contact area, and Question B pertains to temperature change according to the amount of solution 
and temperature difference between two materials. The science questions A and B were descriptive questions that 
required students to provide the reasons for their chosen answer. For each question, students were asked to select 
one or more of the provided answers and write down the rationale behind their selection in order to gain a closer 
understanding of the students’ preconceptions. The last option within each question was designed to allow for 
an “another opinion,’ giving students the opportunity to express opinions divergent from the provided answers. 
Students were given sufficient time to peruse the question and formulate their answer. Right after trying to solve 
the question, the students were asked to rate their confidence level in their answer. Subsequent to these ratings, 
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the correct answers were revealed to the students. At this time, students whose answers matched the correct an- 
swer were classified into the “expected outcome” group, while students whose answers did not match the correct 
answer were classified into the “violation outcome” group. Immediately after presenting the correct answer for each 
question, state curiosity was assessed. The same procedure, from presenting the science question to assessing their 
state curiosity, was repeated for the other question. Considering that the order of question presentation might 
influence the outcome, students in six classes were exposed to questions in the order of “question A — question 
B,’ while students in another seven classes were exposed to questions in the order of “question B — question A.’A 
3-minute break was provided between questions to mitigate the cognitive load stemming from question-solving. 
These procedures were not time-limited and were executed in a manner akin to a standard classroom setting. 
All procedures were conducted by the author to mitigate the possibility of variation in students’ responses and, 
consequently, in the study results, which could arise due to different teachers carrying out the procedures. After 
all the treatment was completed, learning materials containing solutions to the questions were provided, allowing 
participants to learn. Participants were thoroughly briefed on the study's objectives and the significance of their 
involvement, and all results would be treated anonymously and confidentially. 


Figure 2 
Procedure of Assessments and Treatments 


: I 
: I Trying to I ee 4 
Students . ' answer the question | 1 Checking 
reactions | * (expecting the answer) | 1 the answer , 

a 


kaa “sea 


Presenting 
correct answer 


Presenting 
science question 


“etme 


4 . 


+ Interest i 
- Need for cognition i (repeat once for the other question: A+B or B>A) 
- Science Self—CONCEDt  |Reemanaeeeennatenieeaataaiieneie aaa anieninsaaaaaineataaaaaaiaan 


Instrument 


A total of six variables were measured: interest, need for cognition, and confidence as control variables; science 
curiosity as the independent variable; state curiosity as the dependent variable; and science self-concept as the 
moderating variable. When modifying certain items, consultation was conducted with five experts and scholars 
in their field. Additionally, pilot tests were conducted with 5th to 6th-grade primary school students to confirm if 
any issues arose in comprehending and responding to the modified items. All items on the measures in this study 
were designed to be responded to using a 5-point Likert scale ranging from not agree at all to strongly agree. 


Control Variables Measures 


The interest questionnaire consisted of items asking how interested students are in the content of questions 
Aand B. Interest in the content of each question was assessed by three items, with a sample item being “How does 
heat transfer?” Cronbach's a was .880 for question A and .886 for question B. The measurement of the need for 
cognition utilized Lee’s (2004) version of the Need for Cognition Scale, originally developed for college students 
by Cacioppo et al. (1984), which was adapted to be comprehensible for primary school students. Sample items 
are “| enjoy problems that require a lot of thinking,’ and “I like to think seriously for a long time.’ Cronbach's a for 
the total 18 items was .891. The confidence questionnaire comprised two items assessing students’ confidence in 
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their answers after solving questions A or B, with one example being, “I believe the answer | chose will be correct.” 
Cronbach's ain this study was .759. 


Science Curiosity Measure 


The Epistemic Curiosity Scale developed by Litman and Spielberger (2003) was used by modifying certain 
item content to make it a questionnaire of epistemic curiosity about science suitable for primary school students. 
An example of a modified item is as follows: “When | learn new scientific content, | would like to find out more 
about it.’ The measurement of science curiosity in this study consisted of 5 items measuring diversive curiosity and 
5 items measuring specific curiosity. Cronbach’s a for the total 10 items was .943. 


Science Self-Concept Measure 


To assess science self-concept, the items used to measure academic self-concept were adapted from the 
self-concept measurement developed by Kim (1984). These items were modified to gauge self-concept within the 
academic discipline of science (i.e., science self-concept), and subsequently utilized in this study. The question- 
naire consists of 11 items measuring competence and 9 items measuring incompetence. Sample items for this 
measurement included “I can do better science than other students if | set my mind to it” and “I feel like I’m not 
good at science” (reverse coded). Cronbach’s a for the total 20 items was .950. 


State Curiosity Measure 


The Science State Curiosity and Anxiety Scale (SSCAS), developed by Kang et al. (2020), was used to assess 
students’ state curiosity when they check the correct answers to science questions after solving them. The SSCAS 
is appropriately designed for young adolescents and is user-friendly both in administration and scoring. It classifies 
science learning into three phases: “confronting science questions,’ “checking the answer,’ and “learning science 
concepts,’ offering the advantage of assessing state curiosity and state anxiety in each phase. It comprises three sets 
of self-report measures, each comprising 10 items. Within each of the three phases, there are 5 items designed to 
assess science state curiosity and another 5 items measuring science state anxiety. Among them, 5 items measur- 
ing science state curiosity were used in “checking the answer” phase, such as “I want to know why this result came 


out.’ Cronbach's a for the 5 items was .90 for question A and 0.880 for question B. 
Data Analysis 


The data collected for this study was analyzed as follows. First, descriptive statistics such as means and standard 
deviations (SDs) were computed to confirm the levels of the variables used in the analysis. Furthermore, correlation 
analysis was conducted to confirm the relationships between the variables. In order to test the hypothesis that the 
relationship between science curiosity and state curiosity would be moderated by science self-concept when faced 
with an outcome violating expectations, a moderation analysis was conducted using Model 1 of the PROCESS macro 
for SPSS (Hayes, 2013). To operationalize this, the participants were classified into two distinct groups based on 
their encountered outcomes: the violation outcome group (experimental group), which encountered the outcome 
that deviated from their expectations, and the expected outcome group (comparison group), which encountered 
the outcome that was consistent with their expectations. Control variables were grade, gender, question type, 
interest, need for cognition, and confidence in one’s answer. The PROCESS macro model offers the advantage of 
automatically providing mean centering for both the independent and moderating variables, thus mitigating 
multicollinearity. Additionally, it allows for a detailed verification of the significance of the simple slope—i.e., the 
effect of the independent variable on the dependent variable according to the moderating variable (Hayes et al., 
2017). When the moderating effect was significant, each conditional effect was confirmed based on the mean 
value + 1 SD (high level), the mean value (medium level), and the mean value — 1 SD (low level) of the moderating 
variable. Furthermore, the Johnson-Neyman technique identified the range of values for the moderating variable 
that exhibited a significant moderating effect and visualized it as a graph. All statistical analyses were performed 
using SPSS 22.0 and PROCESS macro-3.5.3. 
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Research Results 
Descriptive Statistics and Correlation Analysis 


The results of analyzing the means, SDs, and correlations between the variables for the group facing the 
violation outcome and the group facing the expected outcome are presented in Table 1. In terms of the indepen- 
dent variable, science curiosity, the violation outcome group had a mean of 33.64 (SD = 10.16), and the expected 
outcome group had a mean of 35.43 (SD = 8.82). Regarding the moderating variable, science self-concept, the 
violation outcome group had a mean of 67.28 (SD = 16.53), and the expected outcome group had a mean of 69.82 
(SD = 14.98). In addition, for the dependent variable, state curiosity, the violation outcome group had a mean of 
13.41 (SD = 4.31), and the expected outcome group had a mean of 12.40 (SD = 4.48). 

In both groups, correlation analysis revealed moderate to high positive associations between science curiosity 
and science self-concept (rviolatioin = .710, p < .01; rexpected = .647, p< .01), science self-concept and state curios- 
ity (rviolatioin = .518, p < .01; rexpected = .401, p< .01), as well as science curiosity and state curiosity (rviolatioin = 
.664, p <.01; rexpected = .524, p< .01).In addition, significant positive correlations were observed between interest 
and state curiosity (rviolatioin = .628, p < .01; rexpected = .517, p < .01), as well as between need for cognition and 
state curiosity (rviolatioin = .503, p < .01; rexpected = .541, p< .01). 


Table 1 
Means, Standard Deviations, and Correlations between Variables 


Correlation 
Mean (SD) : iotigd F 
(lower diagonal: violation; upper diagonal: expected) 
: Scale 
Variable 
Violation Expected at 
(N= 242) (N= 242) 1 2 3 4 5 6 7 8 9 
1. Grade 5.54(5.00) 5.48(0.50) -144* 100 .055 034 .158* .059 -.068  .102 
2. Gender 0.45(0.50) 0.40(0.49) 0-1 .004 030 -.195* -.120 -.251%* -314** -180*  -.083 
3. Question type 0.53(0.50) 0.40(0.49) 0-1 -.008  .010 089 -044 -077 015 028 124 
4. Interest 6.71(2.85) 7.38(2.81) 0-12 -.091  -.181** — -.063 463** 337" .566** = .485** = .517** 
5. Need for cognition  60.32(13.14)  62.56(12.14) 1-90 -114 -102 .082 .534* 392" .638** .597** = .541** 
6. Confidence 5.45(1.24) 5.86(1.30) 0-8 068 -.258** §-.018 .233** = .212** A11** = .403** — .384** 
7. Science curiosity 33.64(10.16)  35.43(8.82) 1-50 -.136* -271 005 610% .730** .236** 647" = .524** 
8. Science self-concept 67.82(16.53)  69.82(14.98) 1-100 -.056 -.175** 026 .490** .673** .343** .710** 401** 
9. State curiosity 13.41(4.31)  12.40(4.48) 0-20 -.009 -221** 022 .628** .503** 309% .664** .518** 
Note. *p<.05,**p<.01 


Moderating Role of Science Self-Concept in the Relationship 
between Science Curiosity and State Curiosity when Facing the Violation Outcome 


To verify the moderating effect of science self-concept on the relationship between science curiosity and state 
curiosity, data were analyzed using the PROCESS macro (Model 1) for SPSS, as proposed by Hayes (2013). Verification 
was conducted using bootstrapping with a 95% confidence interval and a sample size of 5000. As shown in Table 
2, the model accounted for 59.4% of the variance in the dependent variable, state curiosity (R?= .594), indicating 
a good model fit (F = 45.984, p < .001). Science curiosity as an independent variable was found to have a positive 
significant effect on state curiosity (B = .192, p < .001). The interaction term between science curiosity and science 
self-concept had a positive significant effect on state curiosity (B = .005, p < .001), and the confidence interval of 
bootstrap [0.003, 0.006] did not include zero. It can be seen that science self-concept moderated the relationship 
between science curiosity and state curiosity (AR?= .051, p < .001). 
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Table 2 


Moderating Effect of Science Self-Concept on the Relationship between Science Curiosity and State Curiosity in Violation 
Outcome Group (N = 293) 


B SE F R? (AR?) 
Grade 651 332 
Gender 123 354 
Control Question type 256 331 
variables Interest Ag0** 074 
Need for cognition .006 .020 45.984*** .594 (.051***) 
Confidence 170 .150 
Science curiosity (A) .192*** 029 
Science self-concept (B) 026 016 


AXB .005*** 001 
Note. *p < .05, **p< .01,***p< .001 


The conditional effect of science curiosity on state curiosity according to the level of science self-concept was 
analyzed, and the results are presented in Table 3. The conditional effect was found to be significant for all three 
conditions: science self-concept smaller than average (M - 1SD), average (M), and larger than average (M + 1SD). 
The conditional effect was the strongest for high values (+ 1SD) of science self-concept, it was still significant 
for medium values (M) and low values (- 1SD) of science self-concept (B = .272, p < .001; B,, = .192, p < .001; 


M+1SD 


By asp = -113, p < .001). This implies that the higher the science self-concept, the stronger the effect of science 


curiosity on state curiosity. 


Table 3 
Conditional Effects of Science Curiosity at Values of Science Self-Concept in Violation Outcome Group (N = 293) 


95% Cl 
Science self-concept Effect SE 
LLCl ULCI 
M-1SD 13" 032 051 175 
M .192*** 029 .136 249 
M+ 1SD 272** 032 210 335 


Note. ***p < .001; M = mean; SD = standard deviation; LLCI = lower limit confidence interval; ULCI = upper limit confidence 
interval 


Specifically, the significance region of the conditional effect of science curiosity was identified through the 
Johnson-Neyman technique. The conditional effect was significant when the science self-concept was higher than 
41.82. 91.468% of all cases had a science self-concept level of 41.82 or higher. Consequently, the moderating ef- 
fect of science self-concept was significant in 91.468% of all cases when science curiosity predicted state curiosity. 


1031 


https://doi.org/10.33225/jbse/23.22.1025 LT 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 
MODERATING ROLE OF SCIENCE SELF-CONCEPT IN ELICITING STATE CURIOSITY WHEN ISSN 1648-3898 /Print/ 


CONFRONTING A VIOLATION OUTCOME 
(PP.1025-1037) ISSN 2538-7138 /Online/ 


Figure 3 
Relationship between Science Curiosity and State Curiosity at Different Levels of Science Self-Concept 
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The interaction effect can be plotted using values one standard deviation above and below the mean (Aiken 
& West, 1991). The result of visualizing the relationship between science curiosity and state curiosity according to 
different levels of science self-concept (mean score, and + 1 SD) is shown in Figure 3. In all three levels, when science 
curiosity increased, state curiosity also increased. However, it was found that the higher the science self-concept, 
the steeper the slope of the increase in state curiosity as science curiosity increased. In other words, it can be seen 
that even if the level of science curiosity was the same, state curiosity further increased when science self-concept 
was high. For the cases with science self-concept scores 1 SD below the mean, the association between science 
curiosity and state curiosity is more attenuated compared to cases with science self-concept scores 1 SD above the 
mean. This implies that the effect of science curiosity on state curiosity is weaker for individuals who have lower 
levels of science self-concept compared to those who have higher levels. 


Moderating Role of Science Self-Concept in the Relationship between Science Curiosity and State Curiosity 
When Facing the Expected Outcome 


The model accounted for 43.7% of the variance in the dependent variable, state curiosity (R?= .437), and 
its fit was confirmed (F = 20.019, p < .001). While students who encountered the expected outcome demonstrated 
that science curiosity had a positive significant effect on state curiosity (B = .097, p < .05), the moderating effect of 
science self-concept on the process of arousing state curiosity was not significant (p = .489). 
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Table 4 


Moderating Effect of Science Self-Concept on the Relationship between Science Curiosity and State Curiosity in Expected 
Outcome Group (N = 242) 


B SE F R? (AR?) 
Grade 353 460 
Gender 831 A486 
Control Question type 1.056* 461 
variables Interest A04** 100 
Need for cognition A12*** 026 20.019*** 437 (.001) 
Confidence .558** 201 
Science curiosity (A) .097* .040 
Science self-concept (B) -.022 021 
AXB -.001 .002 


Note. *p < .05,**p< .01, ***p < .001 


Discussion 


As suggested in several previous studies (Grossnickle, 2016; Kang & Kim, 2021a; Loewenstein, 1994), science 
curiosity as a stable aspect was found to have a positive effect on inducing state curiosity. In addition, the results 
of this study suggest that science self-concept moderates the pathway from science curiosity to state curiosity, 
especially when students are confronted with outcomes that deviate from their expectations, as indicated by 
a significant interaction term between science curiosity and science self-concept. State curiosity, as an urgent 
desire to close knowledge gaps, may require a certain degree of perceived competence about oneself in the 
pertinent academic fields, because its primary goal is to reduce knowledge gaps and acquire specific knowledge 
(Loewenstein, 1994; Shin & Kim, 2019). Therefore, in the process of evoking state curiosity among students who 
encountered the violation outcome, it is judged that those with a higher science self-concept tend to experience 
heightened levels of state curiosity. Promoting curiosity can play a crucial role in providing support for science 
education (Jirout, 2020). Given that higher levels of state curiosity induced during learning have been shown 
to positively influence academic performance and achievement (Gruber & Ranganath, 2019; Gruber et al., 2014; 
Kang et al., 2009; Kang & Kim, 2021b), it is important for science educators to acknowledge the significance of 
cultivating and augmenting students’ science self-concept. 

The results may provide support for existing theories pertaining to individuals’ motivational processes. As 
indicated by the result that science self-concept significantly moderated the relationship between science curi- 
osity and state curiosity, students need to have the belief or confidence that they can address their knowledge 
gaps to generate a high level of state curiosity (Deci, 1975; Pekrun, 2019; Peterson & Cohen, 2019). Therefore, the 
results of this study can be explained by the expectancy-value theory, which posits that a student's expectancy 
beliefs in their ability to bridge knowledge gaps, that is, completing specific tasks in a particular domain, affect 
their subsequent academic performance and goal attainment (Atkinson, 1964; Wigfield & Eccles, 2000). In ad- 
dition, emotions in learning can be influenced by proximal factors in a student, such as situational perceptions 
and cognitive appraisals (Pekrun et al., 2007). In order for emotions like state curiosity to be triggered in learn- 
ing, appraisals of controllability are necessary, and the intensity of emotions increases with higher perceived 
controllability (Pekrun et al., 2007). This control-value theory is premised on the idea that appraisals of control 
play a crucial role in the arousal of emotions experienced during learning (Pekrun, 2006). As the appraisal of 
controllability relies on student’s subjective judgments, it is highly probable that characteristics or qualities 
specific to individual students will be influenced in the appraisal. Therefore, students with a higher science self- 
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concept are more likely to have a stronger belief in their ability to control knowledge gaps in specific situations, 
such as when facing a violation outcome. This, in turn, is presumed to lead to a higher level of curiosity aroused 
to resolve such knowledge gaps. 

Meanwhile, the moderating effect of science self-concept was not observed in situations where students 
were confronted with outcomes that aligned with their expectations. This is in contrast to the results in which 
the science self-concept served as a moderating variable when confronted with violation outcomes. For students 
who encountered expected outcomes, the result of the question (i.e., the correct answer) may not have posed a 
challenging task requiring a new understanding. Consequently, there may be no significant moderating effect 
observed between their state curiosity and science self-concept. 

Science self-concept is a key factor for predicting academic achievement in science (Bouchey & Harter, 2005; 
Jansen et al., 2014; Zhang et al., 2022). Meaningful learning in science education involves changing students’ 
existing knowledge structures, which may include misconceptions, into a more accurate understanding of 
scientific concepts (Chi, 2008). In order to grasp new scientific concepts that students are unfamiliar with, they 
should experience curiosity, which is characterized as an individual’s inclination to learn, explore, and compre- 
hend novel information (Kashdan & Steger, 2007). The temporary state curiosity experienced at these moments 
can be influenced by the learner’s more stable science curiosity (Loewenstein, 1994). Students’ curiosity can 
motivate them to seek new information in anomalous situations, offering support in overcoming learning chal- 
lenges and taking intellectual risks (Engel, 2011; Jirout & Klahr, 2012). The role of academic self-concept might 
also be involved in this process. Therefore, as the results of this study indicated, the moderating effect of science 
self-concept was confirmed in the relationship between science curiosity and state curiosity._ 


Limitations 


The results of the study require cautious interpretation due to several limitations. First, the results were 
derived from data collected from 5th and 6th-grade students in a Korean primary school. To generalize these 
findings to a wider range of ages or regional contexts, further studies are needed. Second, as presented in the 
results of the Johnson-Neyman technique, the moderating effect of science self-concept was observed only 
among students whose science self-concept was not extremely low when facing a violation outcome. Conse- 
quently, generalizing the results of this study to students with significantly low levels of science self-concept 
is challenging. Third, these results may not be applicable to students with low confidence in their answers, as 
they were not included in the analysis. Nevertheless, the applicability of these findings to related studies can 
be contemplated in light of the general consensus that a higher self-concept within a particular discipline can 
positively influence the acquisition of novel or unfamiliar knowledge. 


Conclusions and Implications 


This study explores the moderating effect of science self-concept on eliciting state curiosity in science learn- 
ing for 5th and 6th-grade primary school students, with potential applications in diverse educational settings. The 
study claimed that science self-concept can promote students’ state curiosity when they are confronted with a 
violation outcome. The results of the study provide empirical evidence that the higher the science self-concept, 
the greater the positive effect of science curiosity in stimulating state curiosity. Additionally, they offer insights 
into how students’ self-concept in science can influence their engagement and curiosity when encountering 
unexpected results, a phenomenon relevant to students from various cultural and educational backgrounds. 
Considering that presenting scientific phenomena or outcomes that contradict students’ preconceptions in 
teaching scientific concepts is often necessary, and given that state curiosity triggered during the learning has 
a positive impact on academic achievement, it is important for educators and parents to make efforts to develop 
students’ science self-concept. Meanwhile, the moderating effect of science self-concept was not significant when 
no violation outcomes were confronted. Future studies should therefore examine a broader range of conditions 
or situations under which science self-concept can influence the elicitation of state curiosity, beyond instances 
of encountering the violation outcome. 
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Many studies on strategies to arouse state curiosity in learning thus far have focused on external factors 
(situations or stimuli) that can elicit state curiosity, rather than on students themselves who experience state 
curiosity. However, this study focused on students’ self-concept as the subjects experiencing state curiosity, 
rather than on external situations or stimuli that trigger state curiosity. In particular, the analysis focused on 
whether the level of state curiosity experienced by students when they encountered violation outcomes varies 
depending on their science self-concept levels. The findings are meaningful in that they empirically revealed 
the moderating role of science self-concept in eliciting state curiosity, in light of the theoretical definition of 
science self-concept as one’s perceived competence in science disciplines. 

Students with positive academic self-concept are more likely to enjoy and participate in challenging yet 
attainable tasks. Therefore, the findings that science self-concept positively moderates the arousal of state curi- 
osity can be applied not only in the context of encountering violation outcomes but also in situations involving 
challenging tasks. This presents a direction for future research. 

Although curiosity and academic self-concept are known to have a positive effect on stimulating learning 
and are related to persistence in learning, limited empirical research exists on the relationship between state 
curiosity and academic self-concept. The present study may be an initial step toward empirically understand- 
ing the role of science self-concept in arousing state curiosity, which has implications for educational practices 
worldwide. The findings of this study contribute to revealing the role of science self-concept in the fundamental 
mechanism of inducing state curiosity and open the door to future avenues for research on the relationship 
between state curiosity and science self-concept in science learning. 
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patterns and themes within scholarly 
publications to address a research gap 

in the current literature. The publications 
indexed in the SCOPUS database on 
attitudes and STEM were analyzed to 
collect data. 173 articles were chosen for 
analysis. Bibliometric analysis revealed 

an upward trend in research publications 
between 2008-2022, with notable increases 
in 2013 and 2018 and between 2018 and 
2022. Curtin University was the institution 
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during this period. The results also showed 
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Introduction 


Developing students’ attitudes towards STEM courses in applying sci- 
entific knowledge and fundamental problem-solving skills are the primary 
objectives of education in schools. Hence, researchers have placed attitudes 
at the center of their studies. Attitude is one of the affective factors influenc- 
ing student performance in a particular course or subject area (Martynenko 
et al., 2023; Ozel et al., 2013). For this reason, many researchers have studied 
the effects of students’ attitudes toward STEM courses (Aratijo et al., 2022; 
Martynenko et al., 2023). STEM education provides skills increasingly in 
demand in today’s labor market (Ellison & Allen, 2016). Exploring student at- 
titudes toward STEM (science, technology, engineering, and mathematics) is 
important for researchers as it can significantly impact their engagement and 
interest in choosing a career in one of the related fields (Perdana et al., 2021; 
Zhou et al., 2019). People’s thoughts, beliefs, and opinions about science, 
technology, engineering, and mathematics (STEM) are attitudes toward STEM 
(George, 2006; Kurniawan et al., 2019; Musengimana et al., 2021). Personal 
experiences, cultural backgrounds, educational settings, and social factors 
shape these attitudes (George, 2006; Musengimana et al., 2021). Positive at- 
titudes involve being interested, recognizing career potential, appreciating 
practical uses, and supporting STEM education (Osborne et al., 2003; George, 
2006). Negative attitudes include a lack of interest or relevance, perceived 
difficulties, stereotypes, and an unwelcoming environment (George, 2006). 
Therefore, it is crucial to cultivate a positive attitude toward STEM, as it can 
inspire interest, engagement, and creativity, ultimately resulting in progress 
and innovation in the STEM fields (Razali et al., 2018; Tseng et al., 2011; Ven- 
nix et al., 2018). 

Analyzing students’ attitudes toward STEM can provide valuable insights 
into the barriers or biases preventing certain groups from pursuing STEM 
education and careers (Ball et al., 2017; Nasri et al., 2021). By understanding 
these attitudes, evidence-based policies and curricula can be developed that 
align better with student needs and interests. Furthermore, this process can 
help create a supportive environment that empowers students to explore, 
embrace, and excel in STEM fields. In addition, studying student attitudes 
toward STEM can help identify obstacles and challenges in guiding career 
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paths and fostering a future STEM workforce (Musengimana et al., 2021; Rahmayani & Istiyono, 2022). Thus, edu- 
cators who know their students’ attitudes can design instructional methods and content to encourage curiosity, 
interest, and enthusiasm for STEM (Musengimana et al., 2021). 

A positive attitude towards STEM subjects can encourage students to choose careers in related fields (Razali 
et al., 2018; Vennix et al., 2018). Educators and stakeholders can assess students’ attitudes to determine if they pos- 
sess the necessary dispositions for success in STEM careers, such as problem-solving and critical thinking (Hasanah, 
2020; Razali et al., 2018; Rehman & Butt, 2020). By doing so, they can create supportive learning environments and 
interventions that inspire and empower students to succeed in STEM (Vennix et al., 2018). This information can help 
curriculum makers, researchers, and teachers allocate resources, implement initiatives, and evaluate outcomes for 
more effective interventions. Furthermore, students’ STEM attitudes can provide insight into the factors influencing 
their career decisions in STEM fields (Razali et al., 2018; Siani & Dacin, 2018). This data can aid in aligning various 
software to provide targeted guidance and support to students interested in STEM. 

Researchers have made numerous attempts to measure students’ attitudes toward STEM (Anastasiadis & 
Zirinoglou, 2022), developing their attitudes to encourage them to pursue a career in STEM subjects (Nugent et 
al., 2010; Sahin et al., 2014) and providing assessments or data collection tools to measure their attitudes (Guzey et 
al., 2014; Mahoney, 2010). Numerous studies have explored students’ attitudes toward STEM fields in the literature. 
Although research on student attitudes has made considerable progress in STEM, some gaps should be further 
explored. One of these gaps is related to bibliometric studies. Although the literature on attitudes in the STEM 
fields is growing steadily, the current literature lacks research on bibliometric studies related to attitudes and STEM. 
A few studies have been conducted similarly to the present study. No study conducted a bibliometric analysis re- 
garding attitudes in STEM education. For example, LeThiThu et al. (2021) analyzed SCOPUS publications in middle 
school STEM education from 2000 to 2020. Their findings indicated that middle school STEM education research 
has exploded over the past five years, particularly in the United States. The research topics vary and cover various 
issues, including gender, engineering education, curriculum, etc. Recently, the research conducted by Phuong et 
al. (2023) involved analyzing the bibliographic content of research papers on STEM education from 2006 to 2022. 
Their findings showed a significant and consistent increase in research in this area over the past 5 years. They also 
discovered that highly-regarded journals publishing case studies in STEM education are widely read. 

Important insights can be obtained from bibliometric studies by LeThiThu et al. (2021) and Phuong et al. (2023). 
However, these studies have not specifically examined the bibliometric studies regarding the attitudes of students 
or individuals toward STEM fields. Previous analyses have not provided a complete literature overview of students’ 
attitudes toward STEM. With the significant increase in studies on this topic in recent years, it is important to have 
an accurate understanding of the current state of research. Thus, a dearth of research comprehensively analyzing 
the scholarly knowledge about attitudes generated in STEM fields is appearing in the current literature. Because 
of this reason, this research employs bibliometric methodology to identify patterns and themes in attitudes and 
STEM education within scholarly publications to address this deficiency. The current research gaps regarding at- 
titudes, STEM education, and bibliometric studies are addressed in this research. 

Bibliometrics is a field that overlaps with scientometrics and informetrics and measures scientific research 
development by analyzing published literature (Waltman et al., 2010). VOS-Viewer has been a widely used soft- 
ware for creating and displaying bibliometric networks based on citations, bibliographic links, co-citations, and 
co-authorships (van Eck & Waltman, 2010). This study aims to make a bibliometric analysis of research on attitudes 
in STEM education. The results from this study provide valuable information about researchers, research trends, 
authors, countries, institutions, and publication sources that have contributed to STEM education research in the 
last fifteen years. 


Research Methodology 
General Background 
Bibliometric methods are often utilized in literature searches to enhance understanding of a specific research 
field. This method guides researchers in identifying relevant work and mapping its connections within a field. It allows 
researchers to use data mining techniques that analyze relationships among research, such as the co-occurrence 


of keywords and co-citation networks (van Eck & Waltman, 2010). The researchers can use objective search criteria 
to minimize subjective bias in selecting publications. It is worth noting that bibliometric reviews do not summarize 
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the actual findings of a set of studies but rather examine the bibliographic metadata associated with a group of 
documents to identify trends in knowledge production. Since the literature on attitudes toward science in STEM 
education is relatively high, the descriptive aspects of bibliometric analysis were used in this research. 

Bibliometric analysis is a quantitative evaluation of scientific publications that provides researchers with 
insightful information about a research area or topic based on the various characteristics of existing publications, 
including abstracts, authors, citations, countries, funding sources, keywords, organizations, publishers, and pur- 
poses (Waltman et al., 2010). Due to bibliometric analysis's effectiveness and widespread application in the research 
literature, its techniques are the most highly regarded method for evaluating scientific publications in a particular 
research field or topic (Wang et al., 2018). Bibliometric mapping illustrates the relationships between nations, 
disciplines, subject areas, organizations, specific publications, and authors. Quantifying and assessing a specific 
aspect of research, bibliometric research facilitates the identification of trends in a field. The bibliometric analysis 
enables the monitoring of studies, researchers, institutions, and scientific developments pertinent to a particular 
scientific topic. The analysis of bibliometric studies is increasingly used to evaluate academic disciplines across the 
literature. The bibliometric analysis provided a comprehensive summary of relevant research based on numerous 
criteria. The relationship between relevant objects like authors, countries, sources, affiliations, and documents was 
subsequently determined using co-authorship analysis, citation analysis, and co-occurrence of author keywords 
(Waltman et al., 2010). In this study, researchers used VOSviewer software to analyze the data to comprehensively 
understand the research trends on attitudes and STEM. 


Data Collection 


This analysis was conducted by searching the SCOPUS database for scholarly articles on STEM attitudes. The 
researchers chose SCOPUS because it is one of the bibliographic databases of peer-reviewed literature, scholarly 
journals, publications, and conference proceedings. It offers access to a vast collection of high-quality scientific 
literature, rendering it an indispensable resource. SCOPUS is an essential and useful resource for researchers due 
to its comprehensive coverage of high-quality scientific literature, advanced search capabilities, citation analysis, 
and collaboration tools. It contains over 14,000 indexed journals from a variety of disciplines. Due to these factors, 
researchers utilized it to collect data for bibliometric research and to gain access to exhaustive publications on 
STEM and attitudes. The researchers initially used basic keywords such as“attitude,“attitudes,’and “STEM” to locate 
pertinent publications. Researchers used filtering options to limit their database search results to social science 
publications, journals, and English-only content. In addition, the search covered the years 2008-2022. Consequently, 
222 studies were retrieved from the database. These search terms ((KEY (attitude) OR KEY (attitudes) AND KEY 
(stem)) AND (EXCLUDE (PUBYEAR, 2023)) AND (LIMIT-TO (SUBJAREA, “SOCI")) AND (LIMIT-TO (LANGUAGE, “English”)) 
AND (LIMIT-TO (PUBYEAR, 2022) TO (PUBYEAR, 2008))) were employed to retrieve 222 studies from the database. 

In May 2022, the researchers searched the database. Later, two researchers independently examined all the 
studies. They read all 222 studies identified in the initial database analysis. They excluded studies unrelated to STEM 
attitudes or STEM education during this examination. 49 studies unrelated to the search criteria were excluded. 
Thus, a total of 173 studies were included in the bibliometric analysis. The results of the literature search were 173 
documents. Document types included original articles (n = 157, 90.7%), conference papers (n = 13, 7.51%), book 
chapters (n = 2, 1.15%), and notes (n = 1, 0.05%). Two types of analyses were conducted. Excel (2007; Microsoft) 
initially generated the initial data statistics. The data was processed using VOSviewer. 


Data Analysis 


After obtaining the final search results from the SCOPUS website, the researchers extracted database files for 
the bibliometric analysis. They downloaded BibTeX and CSV files for the bibliometric analysis of 67 publications. 
Later, the researchers converted these files to the TAB file format so that the program for bibliometric analysis 
could use them. Tab-separated (Win) files for bibliometric analysis, in addition to the complete dataset and cited 
references. For the bibliometric analysis of this study, the VOSviewer was used to display the network visualiza- 
tion of the most frequently used keywords, abstract words, citation analysis, and co-citation analysis. The file was 
uploaded to the VOSviewer software. Two researchers conducted the data analysis. 

The researchers used the software because it allowed to analyze literature networks and create visual graph- 
ics. Thus, the researchers performed network analysis using co-citation or co-citation analysis to identify the most 
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influential component based on keywords, authors, sources, and references. This study's co-citation analysis de- 
termines which author, source, or reference is frequently cited. 


Research Results 
Number of Publications 
Figure 1 demonstrates the upward trend in publications between 2008 to 2022. In 2022, 33 publications were 
published. In 2013 and 2018, and between 2018 and 2022, the number of publications increased substantially. In 


2010, no publications were published on the research topic in this study. 


Figure 1 
Number of Publications 
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Table 1 shows the institutions that contributed most to the literature between 2008 and 2022. These insti- 
tutions were Curtin University (n = 8), followed by the National Pingtung University of Science and Technology 
(n = 4), The University of Texas at Austin (n = 4), National Taiwan Normal University (n = 4), and the Michigan State 
University (n = 4), Universiti Kebangsaan Malaysia (n = 3), Zhejiang Normal University (n = 3), NC State University 
(n = 3), South China Normal University (n = 3), and the University of Wyoming (n = 3). The results revealed that 
only Curtin University had contributed to research on STEM attitudes. Regarding attitudes toward STEM education, 
the other institutions from Taiwan, USA, Malaysia, and China did contribute. These institutions were from Chinese, 
Malaysian, and Taiwanese universities. Universities in the United States of America (USA) and Australia have been 
at the forefront of attitude and STEM research. Taiwan was the most significant contributor (see Table 1). 


Table 1 
Top 10 Institutions Contributing to Research on Attitudes and STEM between 2008 and 2022 


Institutions Country Number of Publications 
Curtin University Australia 8 
National Pingtung University of Science and Technology Taiwan 4 
The University of Texas at Austin USA 4 
National Taiwan Normal University Taiwan 4 
Michigan State University USA 4 
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Institutions Country Number of Publications 
Universiti Kebangsaan Malaysia Malaysia 3 
Zhejiang Normal University China 3 
North Carolina State University USA 3 
South China Normal University China 3 
University of Wyoming USA 3 


Table 2 shows that the United States had the most publications (n = 73, or 42.1%). Turkey is the second most 
productive nation (n = 16, 9.2%). Australia ranked third with 15 publications (n = 15, or 8.6%). Taiwan ranks the 
fourth with 11 publications (n = 11, 6.3%). It was found that China was ranked fifth. The following countries are 
Malaysia (n = 8, 4.6%), United Kingdom (n = 8, 4.6%), Netherlands (n =7, 4%), Spain (n =6, 3.4%), Belgium (n = 5, 
2.9%), and Indonesia (n = 5, 2.9%) in terms of numbers of publications on attitudes and STEM. 


Table 2 
Top 10 Contributing Countries 


Countries Number of Publications 
United States 73 
Turkey 16 
Australia 15 
Taiwan 1 
China 10 
Malaysia 8 
United Kingdom 8 
Netherlands 7 
Spain 6 
Belgium 5 
Indonesia 5 


Network Analysis of Sources 


Table 3 presents the top 10 sources identified by co-citation analysis using the VOSviewer software. The In- 
ternational Journal of Science Education (IJSE) emerged with the highest number of citations, surpassing all other 
sources by being cited more than 200 times. A close connection exists between individuals’ views towards Science, 
Technology, Engineering, and Mathematics (STEM) and the research focus of articles published in the International 
Journal of Science Education (IJSE). The International Journal of Science Education (IJSE) achieved a 5-year impact 
factor of 2.776 in 2021. In the cluster analysis, numerous researchers cited articles from the International Journal 
of Science Education, International Journal of STEM Education, School Science and Mathematics, International 
Journal of Technology and Design Education, and Research in Science Education. These journals are long-term 
publications that larger publishers have published for over 30 years. Since 2014, only the International Journal of 
STEM Education has been published. In addition to IJSE, the Journal of Research in Science Teaching was cited over 
150 times, with Science Education receiving the most citations (124 times). 
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Table 3 


Top 10 Sources by Co-Citation 
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International Journal of Science Education 


Journal of Research in Science Teaching 


Science Education 


International Journal of STEM Education 


Journal of Personality and Social Psychology 


School Science and Mathematics 


International Journal of Technology and Design Education 


Journal of Educational Psychology 


Computers & Education 


Research in Science Education 


Network Analysis of Authors 


Citations 


202 
168 
124 
102 
92 
92 
90 
86 
78 
69 


Total link Strength 


2793 
2701 
2177 
1335 
1541 
1357 
1289 
1387 
984 
1108 


2008-2022 
(PpP.1038-1049) 


Cluster 


oa oO 


Table 4 lists the 10 most co-cited authors. Only Barry J. Fraser and Jaquelynne S. Eccles were cited more than 
one hundred times. After these two authors, eight other authors were co-cited over fifty times, including Jonathan 
Osborne, Louise Archer, Albert Bandura, Tamara J. Moore, Jennifer Dewitt, Allan Wigfield, and Gillian H. Roehrig. 

Table 5 displays the top ten references based on co-citation analysis. The only reference co-cited more than 
ten times is Osborne et al. (2003). Four references, including Breiner et al. (2012), Wang (2013), Kelley and Knowles 
(2016), and Unfried et al. (2015), were cited more than 5 times. Two of the top 10 references based on co-citation 
were from School Science and Mathematics. 


Table 4 


Top 10 Authors by Co-Citation 


oO a nN DO oO FF WwW YP — 


= 
oO 


Barry J. Fraser 
Jaquelynne S. Eccles 
Jonathan Osborne 
Louise Archer 

Albert Bandura 
Tamara J. Moore 
Jennifer Dewitt 

Allan Wigfield 

Gillian H. Roehrig 
Robert H. Tai 


https://doi.org/10.33225/jbse/23.22.1038 


Organization 


Curtin University 
University of California 
King’s College London 
University College London 
Standford University 
Purdue University 
University College London 
University of Maryland 


University of Minnesota 


University of Virginia 


Country 


Australia 

United States 
United Kingdom 
United Kingdom 
United States 
United States 
United Kingdom 
United States 
United States 
United States 


Citations ehanih 
124 698 
107 1376 
86 1441 
67 1337 
62 793 
62 467 
60 1243 
60 818 
52 473 
47 572 


Cluster 
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Table 5 
Top 10 References by Co-Citation 
Authors(s) Source Citations Total tink Cluster 
Strength 
1 Attitudes towards science: A review of the Osborne et al. International Journal of 13 7 D 
literature and its implications (2003) Science Education 
9 What is STEM? A discussion about concep- Breiner et al. School Science and 7 8 9 
tions of STEM in education and partnerships (2012) Mathematics 
Why students choose STEM majors: Motiva- : 4 
; : : American Educational 
3 tion, high school learning, and postsecondary Wang (2013) 7 5 1 
Research Journal 
context of support 
4 A conceptual framework for integrated STEM _ Kelley & Knowles __ International Journal of 6 7 9 
education (2016) STEM Education 
The development and validation of a measure Uinffied etal. Joumal of Psychoedu- 
5 _ of student attitudes toward science, technol- (2015) Le ee 6 6 1 
ogy, engineering, and math (S-STEM) 
Attitudes towards science, technology, aiaeial International Journal of 
6 — engineering and mathematics (STEM) in a (201 3) f Technology and Design 5 9 2 
project-based learning (PUBL) environment Education 
; ro sd Bybee Technology and Engi- 
7 Advancing STEM education: A 2020 vision (2010) neering Teacher 5 8 1 
3 Learning for stem literacy: STEM literacy for Zollman School Science and 5 6 9 
learning (2012) Mathematics 
9 A conceptual framework for integrated stem Kelley & Knowles _ International Journal of 5 5 1 


education (2016) STEM Education 


Pipeline persistence: Examining the associa- P 
: . , : Maltese & Tai : : 

10 tion of educational experiences with earned (2011) Science Education 5 5 1 
degrees in STEM among US students 


The cluster analysis results indicated that the top 10 references by co-citation were frequently cited together. 
The findings for this result produced four clusters, including the top 10 references. The studies of Wang (2013), Tseng 
et al. (2013), Maltese and Tai (2011) comprised the first cluster. The second cluster consisted of Breiner et al. (2012), 
Kelley and Knowles (2016), and Zollman (2012) studies. The third cluster was comprised of the research conducted 
by Bybee (2010), Maltese and Tai (2011), and Unfried et al. (2015). The fourth cluster contained the Osborne et al. 
(2003) article. The frequency analysis revealed that the 10 most frequently occurring keywords were associated with 
the following topics: STEM, attitudes, STEM education, students, attitude, STEM (science, technology, engineering, 
and mathematics), gender, STEM attitudes, engineering education, and education (see Table 6). 
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Table 6 


Ten Most Commonly Used Keywords 


Keyword Occurrences Total Link Strength 
STEM 64 72 
Attitudes 33 35 
STEM education 32 32 
Students 23 46 
Attitude 20 33 
STEM (science, technology, engineering and mathematics)” 15 33 
Gender 13 19 
STEM attitudes 13 6 
Engineering education 12 35 
Education 11 22 
Teaching 10 23 


Based on a cluster analysis, the keywords have been grouped into six natural clusters (refer to Figure 2 and 
Table 7). Keywords appearing in the same cluster are commonly found in the research literature. Each cluster can 
indicate multiple hotspots for research. Although the keywords from different clusters differ, some research litera- 
ture may address the same research objective that different clusters point to. By analyzing each cluster’s keywords, 
readers can better understand the research hotspots regarding attitudes and STEM. Cluster 1 primarily examined 
students’ attitudes and the effects of their genders on their attitudes toward STEM or STEM attitudes. 


Table 7 
Clusters of Keywords 
Number Cluster name Main keywords 
1 Attitude Attitude, attitudes toward science, female, gender, human, human experiment, male, mathematics, motivation, 
science, stereotype 
9 STEM Attitudes, computational thinking, education computing, integrated STEM, middle school, secondary school, self- 
efficacy, STEM, STEM attitude, STEM attitudes, STEM career interest, STEM education 
Engineering education, learning attitude, learning attitudes, STEM (science, technology, engineering and math- 
Students : . 
3 ematics), student attitudes, students, surveys 
4 Education Education, learning, perception, public attitude, student, sustainability, teaching 
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Figure 2 


A Visualization of Keywords’ Networks 
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Discussion 


This research aimed to employ a bibliometric analysis of research on attitudes in STEM education to identify 
patterns and themes within scholarly publications to address a research gap in the current literature. First, the results 
revealed an upward publication trend from 2008 to 2022. In particular, in 2013 and 2018, and between 2018 and 
2022, the number of publications increased substantially. Regarding the increase in the number of research in STEM 
education, this result is remarkably similar to that of Phuong et al. (2023), who found a similar increase in the number 
of implementations of STEM education. This result is similar to LeThiThu et al. (2021), who found that a growing trend 
of STEM education in middle school can be divided between 2016 and 2020. 

Curtin University was the institution that contributed the most to the literature between 2008 and 2022. Na- 
tional Pingtung University of Science and Technology (n= 4), The University of Texas at Austin (n = 4), National Taiwan 
Normal University (n = 4), and Michigan State University were the other institutions that contributed the most to the 
research on attitudes toward STEM during the last fifteen years. When the results in Table 1 are examined, it can be 
concluded that the institutions from Australia, Taiwan, and the USA contributed equally to the number of research 
on attitudes and STEM between 2008 and 2022. This result is remarkably similar to that of Phuong et al. (2023) and 
LeThiThu et al. (2021). Phuong et al. (2023) found that seven of the 10 productive were from the USA, while LeThiThu 
et al. (2021) studied two decades of STEM education in the middle school level and found that the top 10 productive 
institutions were from the USA. 

The results demonstrated that the United States had the most publications on attitudes and STEM. Turkey is the 
second most productive nation, while Australia ranked third. Taiwan ranks fourth with 11 publications, and China 
is ranked fifth. The following countries are Malaysia, the United Kingdom, the Netherlands, Spain, Belgium, and 
Indonesia. This result is remarkably similar to that of Phuong et al. (2023), who discovered that seven of the top 10 
best-performing authors are from the United States, while the remaining three are from Ireland and Turkey. LeThiThu 
et al. (2021) found a similar finding, revealing that the top countries were the USA, Turkey, and Australia. 

The USE had the most citations, with The Journal of Research in Science Teaching and Science Education closely 
behind. This finding aligns with LeThiThu et al’s (2021) results. Notably, the top 10 journals in co-citation analysis are 
focused on educational research, technology, engineering education, science, and STEM education and are considered 
prestigious and influential. There could be several reasons for this outcome. Firstly, highly-regarded journals have 
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stricter peer-review processes to ensure that they publish only high-quality articles. Secondly, these publications 
release more issues and articles than other periodicals. Additionally, we observed that scientists from the United 
States, the United Kingdom, and Australia wrote more articles. Six of the top 10 authors based on co-citation analysis 
were from the USA. 

Another result also revealed that Barry J. Fraser and Jaquelynne S. Eccles were cited more than one hundred times 
in the publications. These authors came from Australia and the United States, respectively. Similarities exist between 
this result and Phuong et al. (2023) and LeThiThu et al. (2021). Looking at these results, we can expect Australia and 
the United States to have a strong presence in creating knowledge about STEM education through scholarly par- 
ticipation. It is interesting to note that the top 10 references were often cited together. Finally, the results regarding 
the frequency analysis revealed that the 10 most frequently occurring keywords were associated with the following 
topics: stem, attitudes, stem education, students, attitude, stem (science, technology, engineering, and mathematics), 
gender, stem attitudes, engineering education, and education. 


Conclusion and Implications 


Although STEM research has received growing attention in the educational literature, very little information about 
the current status of research on attitudes toward STEM using the bibliometric analysis method is known. The present 
research is the first bibliometric study that examined the current state of attitudes and STEM research. Consequently, 
this study contributes to the existing body of knowledge by providing an overview of the current state of research 
in this field. The research was analyzed to determine the most prominent authors, countries, sources, institutions, 
and publications within the past 15 years. The primary focus was on examining the interconnectedness of sources, 
authors, references, and keywords that contribute to the overall productivity in this field. Based on the findings, it 
can be deduced that Curtin University emerged as the most productive academic institution. At the same time, the 
United States demonstrated the highest level of productivity in research about attitudes and STEM. This result dem- 
onstrates the significant contributions made by the United States academics. It is anticipated that the United States 
will continue to lead in this field of research. Based on this result, exploring the state of research in other countries is 
important. Future studies should explore the attitudes in STEM education in other countries. 

Together with the JRST and Science Education, the IJSE journals provide a forum for high-quality publications 
in this field. The USE was the source with the most citations for research on attitudes and STEM. Regarding research 
on attitudes in STEM, renowned academics had strong collaborations, such as between the groups led by Fraser and 
Eccles, from institutions in the United States and Australia. Based on this finding, these international academic col- 
laborations are anticipated to be strengthened in light of rising STEM-related and attitude-related concerns. 

This study's analysis was conducted using studies indexed in the SCOPUS database. Therefore, future research 
may consider incorporating data from other databases, such as ERIC, EBSCOhost, and Web of Science. Future research 
can track this field's development and international cooperation networks between various nations and organizations. 
Future academic studies can also examine changes in research on student attitudes toward STEM or STEM fields. As 
STEM advances rapidly, researchers often need to provide updates on bibliometric research related to attitudes in 
STEM education. This analysis helps keep track of the field's progress. To stay up-to-date with the latest research in 
STEM education, it is important to perform periodic bibliographic analysis. 


Limitations 


Even though this study provided a comprehensive knowledge map of research on attitudes and STEM, several 
limitations should be noted. Although this study offers valuable insights into STEM attitudes, it is important to note 
that the limited time frame used for the search may have affected the breadth and depth of the studies included. 
While this bibliometric review aimed to provide a thorough overview of research on STEM education attitudes, further 
studies are needed to conduct a more detailed analysis. Furthermore, there are still some unanswered questions 
regarding the effectiveness of strategies, methods, and techniques used to influence student attitudes toward STEM 
education in our search strategy. As a result, there is an urgent need for further research on the impact, design, and 
application of various technologies and online learning strategies to better understand their influence on student 
attitudes toward STEM education. 

The search parameters used in this research, which included only SCOPUS-provided and English-language 
documents, might exclude relevant studies unavailable in SCOPUS or written in other languages. It is noteworthy 
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that while SCOPUS is widely recognized as the greatest multidisciplinary database for scientific literature and encom- 
passes conference proceedings in the form of journal articles, it is possible that certain conference proceedings about 
student recruiting and STEM fields may not be encompassed within its scope. Hence, it is essential to comprehend 
that these constraints can impact the results of this research. Bibliometric studies should consider broadening their 
search parameters to encompass supplementary scholarly inquiries. 


Acknowledgments 


This research was supported by the Zhejiang Province Philosophy and Social Science Planning Project 
(No.23NDJC145YB). 


Declaration of Interest 
The authors declare no competing interest. 
References 


Anastasiadis, L., & Zirinoglou, P. (2022). Evaluating students’ attitudes toward STEM education: The case of Greek pre-service teachers. 
International Journal of Education and Social Science, 9(4), 35-53. https://www.researchgate.net/publication/364282752_ 
Evaluating_Students’_Attitudes_Toward_STEM_Education_The_Case_of_Greek_Pre-_Service_Teachers 

Araujo, J. L., Morais, C., & Paiva, J. C. (2022). Citizen science as a pedagogical tool in chemistry education: Students’ 
attitudes and teachers’ perceptions. /nterdisciplinary Journal of Environmental and Science Education, 18(2), Article 
€2271. https://doi.org/10.21601/ijese/11841 

Ball, C., Huang, K.T., Cotten, S. R., & Rikard, R. (2017). Pressurizing the STEM pipeline: An expectancy-value theory analysis of youths’ 
STEM attitudes. Journal of Science Education and Technology, 26(4), 372-382. https://doi.org/10.1007/s10956-017-9685-1 

Breiner, J. M., Harkness, S. S., Johnson, C. C., & Koehler, C. M. (2012). What Is STEM? A discussion about conceptions of STEM in 
education and partnerships. School Science and Mathematics, 112(1), 3-11. https://doi.org/10.1111/j.1949-8594.2011.00109.x 

Bybee, R. W. (2010). Advancing STEM education: A 2020 vision. Technology and Engineering Teacher, 70(1), 30-35. 

Ellison, S., & Allen, B. (2016). Disruptive innovation, labor markets, and Big Valley STEM School: Network analysis in STEM education. 
Cultural Studies of Science Education, 13(1), 267-298. https://doi.org/10.1007/s11422-016-9786-9 

George, R. (2006). A cross-domain analysis of change in students’ attitudes toward science and attitudes about the utility of science. 
International Journal of Science Education, 28(6), 571-589. https://doi.org/10.1080/09500690500338755 

Guzey, S. S., Harwell, M., & Moore, T. (2014). Development of an instrument to assess attitudes toward science, technology, 
engineering, and mathematics (STEM). School Science and Mathematics, 114(6), 271-279. https://doi.org/10.1111/ssm.12077 

Hasanah, U. (2020). Key definitions of STEM education: Literature review. Interdisciplinary Journal of Environmental and Science 
Education, 16(3), Article e2217. https://doi.org/10.29333/ijese/8336 

Kelley, T. R., & Knowles, J. G. (2016). A conceptual framework for integrated STEM education. /nternational Journal of STEM Education, 
3(1), Article 11. https://doi.org/10.1186/s40594-016-0046-z 

Kurniawan, D. A,, Astalini, A., Darmaji, D., & Melsayanti, R. (2019). Students’ attitude towards natural sciences. International Journal 
of Evaluation and Research in Education (ERE), 8(3), 455. https://doi.org/10.11591/ijere.v8i3.16395 

Le Thi Thu, H., Tran, T., Trinh Thi Phuong, T., Le Thi Tuyet, T., Le Huy, H., & Vu Thi, T. (2021). Two decades of STEM education 
research in middle school: A bibliometrics analysis in Scopus database (2000-2020). Education Sciences, 11(7), Article 353. 
https://doi.org/10.3390/educsci1 1070353 

Mahoney, M. P. (2010). Students’ attitudes toward STEM: Development of an instrument for high school STEM-based programs. Journal 
of Technology Studies, 36(1), 24-34. https://files.eric.ed.gov/fulltext/EJ906158.pdf 

Maltese, A. V., & Tai, R. H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees 
in STEM among U.S. students. Science Education, 95(5), 877-907. https://doi.org/10.1002/sce.20441 

Martynenko, O. O., Pashanova, O. V., Korzhuey, A. V., Prokopyey, A. |., Sokolova, N. L., & Sokolova, E. G. (2023). Exploring attitudes 
towards STEM education: A global analysis of university, middle school, and elementary school perspectives. Eurasia Journal 
of Mathematics, Science and Technology Education, 19(3), Article em2234. https://doi.org/10.29333/ejmste/12968 

Musengimana, J., Kampire, E., & Ntawiha, P. (2021). Factors affecting secondary schools students’ attitudes toward learning 
chemistry: A review of literature. Eurasia Journal of Mathematics, Science and Technology Education, 17(1), Article em1931. 
https://doi.org/10.29333/ejmste/9379 

Nasri, N., Rahimi, N. M., Nasri, N. M., & Talib, M. A. A. (2021). A comparison study between universal design for learning-multiple 
intelligence (UDL-Ml) oriented STEM program and traditional STEM program for inclusive education. Sustainability, 13(2), 
Article 554. https://doi.org/10.3390/su1 3020554 

Nugent, G., Barker, B., Grandgenett, N., & Adamchuk, V. I. (2010). Impact of robotics and geospatial technology 
interventions on youth STEM learning and attitudes. Journal of Research on Technology in Education, 42(4), 391-408. 
https://doi.org/10.1080/15391523.2010.10782557 


1048 


WSS” https://doi.org/10.33225/jbse/23.22.1038 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


ISSN 1648-3898 /Print/ A BIBLIOMETRIC ANALYSIS OF LITERATURE ON ATTITUDES IN STEM EDUCATION IN 
2008-2022 
ISSN 2538-7138 /oniine/ (pp.1038-1049) 


Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International 
Journal of Science Education, 25(9), 1049-1079. https://doi.org/10.1080/0950069032000032199 

Ozel, M., Caglak, S., & Erdogan, M. (2013). Are affective factors a good predictor of science achievement? Examining the role of affective 
factors based on PISA 2006. Learning and Individual Differences, 24(1), 73-82. https://doi.org/10.1016/j.lindif.2012.09.006 

Perdana, R., Apriani, A. N., Richardo, R., Rochaendi, E., & Kusuma, C. (2021). Elementary students’ attitudes towards STEM 
and 21st-century skills. International Journal of Evaluation and Research in Education (JERE), 10(3), Article 1080. 
https://doi.org/10.11591/ijere.v10i3.21389 

Phuong, N.L., Hien, L.T.T., Linh, N. Q., Thao, T.T. P., Pham, H.-H. T., Giang, N.T., & Thuy, V. T. (2023). Implementation of STEM education: 
A bibliometrics analysis from case study research in Scopus database. Eurasia Journal of Mathematics, Science and Technology 
Education, 19(6), Article em2278. https://doi.org/10.29333/ejmste/13216 

Rahmayani, F., &Istiyono, E. (2022). Affective assessment instrument to assess student attitudes towards science, technology, engineering 
and mathematics. Journal of Education Research and Evaluation, 6(4), 637-644. https://doi.org/10.23887/jere.v6i4.47681 

Razali, F., Talib, O., Manaf, U.K. A., & Hassan, S. A. (2018). Students’ attitude towards science, technology, engineering and mathematics 
in developing career aspirations. International Journal of Academic Research in Business and Social Sciences, 8(5), 946-960. 
https://doi.org/10.6007/ijarbss/v8-i5/4242 

Rehman, A., & Butt, |. H. (2020). Elementary school female students’ attitude towards STEM. Journal of Business and Social Review in 
Emerging Economies, 6(2), 511-515. https://doi.org/10.267 10/jbsee.v6i2.1161 

Sahin, A., Gulacar, O., & Stuessy, C. (2014). High school students’ perceptions of the effects of the International Science Olympiad 
on their STEM career aspirations and twenty-first-century skill development. Research in Science Education, 45(6), 785-805. 
https://doi.org/10.1007/s11165-014-9439-5 

Siani, A., & Dacin, C. (2018). An evaluation of gender bias and pupils’ attitude towards STEM disciplines in the transition 
between compulsory and voluntary schooling. New Directions in the Teaching of Physical Sciences, 13(1), 1-17. 
https://doi.org/10.29311/ndtps.v0i13.2966 

Tseng, K. H., Chang, C. C., Lou, S. J., & Chen, W. P. (2011). Attitudes towards science, technology, engineering and mathematics 
(STEM) in a project-based learning (PjBL) environment. International Journal of Technology and Design Education, 23(1), 87-102. 
https://doi.org/10.1007/s10798-01 1-9160-x 

Unfried, A., Faber, M., Stanhope, D. S., & Wiebe, E. (2015). The development and validation of a measure of student attitudes 
toward science, technology, engineering, and math (S-STEM). Journal of Psychoeducational Assessment, 33(7), 622-639. 
https://doi.org/10.1177/0734282915571160 

van Eck, N. J., & Waltman, L. (2009). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 
84(2), 523-538. https://doi.org/10.1007/s11192-009-0146-3 

Vennix, J., den Brok, P., & Taconis, R. (2018). Do outreach activities in secondary STEM education motivate students 
and improve their attitudes towards STEM? /nternational Journal of Science Education, 40(11), 1263-1283. 
https://doi.org/10.1080/09500693.2018.1473659 

Waltman, L., Van Eck, N.J., & Noyons, E.C.M. (2010). A unified approach to mapping and clustering of bibliometric networks. Journal 
of Informetrics, 4(4), 629-635. https://doi.org/10.1016/j.joi.2010.07.002 

Wang, X. (2013). Why students choose STEM majors. American Educational Research Journal, 50(5), 1081-1121. 
https://doi.org/10.3102/0002831213488622 

Wang, P,, Zhu, F., Song, H., & Hou, J. (2018). A bibliometric retrospective of the Journal Eurasia Journal of Mathematics, Science 
and Technology Education between 2012 and 2017. Eurasia Journal of Mathematics, Science and Technology Education, 14(3), 
765-775. https://doi.org/10.12973/ejmste/80911 

Zhou, S.N., Zeng, H., Xu, S. R., Chen, L. C., & Xiao, H. (2019). Exploring changes in primary students’ attitudes towards science, 
technology, engineering, and mathematics (STEM) across genders and grade levels. Journal of Baltic Science Education, 18(3), 
466-480. https://doi.org/10.33225/jbse/19.18.466 

Zollman, A. (2012). Learning for STEM literacy: STEM literacy for learning. School Science and Mathematics, 112(1), 12-19. 
https://doi.org/10.1111/j.1949-8594.2012.00101.x 


Received: August 10, 2023 Revised: October 31, 2023 Accepted: November 18, 2023 


Cite as: Ma, K., & Hui, B.-H. (2023). A bibliometric analysis of literature on attitudes in STEM education in 2008-2022. Journal 
of Baltic Science Education, 22(6), 1038-1049. https://doi.org/10.33225/jbse/23.22.1038 


Ke Ma Master of Education (M.E.), Zhejiang SCI-TECH University, Zhejiang 
province, China. 
mail: 21009024@qq.com 


Bei-He Hui PhD, Zhejiang SCI-TECH University, Zhejiang province, China. 
(Corresponding author) mail: chinahbh@126.com 
ORCID: https://orcid.org/0009-0003-6917-093X 


1049 


https://doi.org/10.33225/jbse/23.22.1038 QT 


JOURNAL 
OF®BALTIC 
SCIENCE 
EDUCATION 


ISSN 1648-3898 feiss 
ISSN 2538-7138 ionines 


Abstract. Nowadays, the assessment 

of student performance has become 
increasingly technology-based, a trend that 
can also be observed in the evaluation of 
scientific reasoning, with more and more 
of the formerly paper-based assessment 
tools moving into the digital space. The 
study aimed to examine the reliability 

and validity of the paper-based and 
computer-based forms of the Science-K 
Inventory, which assesses children’s 
scientific reasoning in three aspects: 
experimentation, data interpretation, and 
understanding of the nature of science. 

The pilot study involved 84 fourth-grade 
Hungarian students, with 39 students 
taking the paper-based test and 45 
students taking the computer-based test. 
Rasch measurements and reliability tests 
have indicated that both the paper-based 
and computer-based test versions are 
equally valid for assessing the scientific 
reasoning skills of fourth graders. Students 
achieved high test scores in both mediums, 
and there were no significant differences 
between boys’ and girls’ scientific reasoning 
in either test type. The novelty of this 
research was that the Science-K Inventory 
had not yet been tested in a computer- 
based format. The results demonstrate that 
the Science-K Inventory can be effectively 
utilized in digital testing to provide teachers 
with rapid and valuable information for 
fostering the development of their students’ 
scientific reasoning. 

Keywords: computer-based testing, 
paper-based testing, primary school, 
Science-K Inventory, scientific reasoning 
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Introduction 


Science education in the 21st century seeks to develop scientific literacy 
that will enable society to deal effectively with technological progress, the 
rapid and sometimes uncontrolled spread of information and global chal- 
lenges. This requires, in addition to disciplinary knowledge, appropriate 
scientific thinking and inquiry skills, as well as an understanding of the nature 
of science. Scientific thinking is the set of mental processes that are used to 
solve scientific problems, acquire scientific knowledge, carry out investiga- 
tions and reflect on research results (Dunbar & Fugelsang, 2005; Dunbar & 
Klahr, 2012; Kuhn, 2011). The domain-specific strand of research on scientific 
thinking focuses on children’s conceptual development and conceptual 
change in different disciplinary domains, while the domain-general strand 
focuses more on identifying the reasoning processes involved in the acquisi- 
tion of scientific knowledge (Zimmerman, 2007; Zimmerman & Klahr, 2018). 
The current study is concerned with the latter research area. 

Scientific reasoning plays a central role in the acquisition of scientific 
literacy. Higher-order reasoning skills are needed to solve problems and make 
decisions, just as they are needed to understand complex concepts or the 
nature of science (Lawson, 2004). Research on the development of reasoning 
skills is rooted in the work of Piaget (Inhelder & Piaget, 1958), who investigated 
children’s reasoning operations in various tasks (e.g., pendulum, balancing) 
and the process by which children systematically explore the world: formulat- 
ing hypotheses, predicting outcomes, manipulating variables, observing, and 
making inferences. While early research (see, for example, Inhelder & Piaget, 
1958; Kuhn et al., 1988) suggested that the acquisition of scientific reasoning 
skills could not begin until adolescence, recent research suggests that some 
skills can be developed earlier (Zimmerman, 2007). There is growing evidence 
that primary school students have a wide range of scientific reasoning skills: 
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by the age of 8 years, students prefer controlled experiments over experiments with confounding features (Bullock 
et al., 2009); they can distinguish between hypotheses and evidence (Sodian et al., 1991); children have the ability 
to understand the control-of-variables strategy (CVS) (Peteranderl & Edelsbrunner, 2020); they can draw inferences 
from graphical data (Koerber & Sodian, 2009); and they have a rudimentary understanding of the Nature of Science 
(NOS) (Koerber et al., 2015). 

Although scientific reasoning is rooted in the intuitive information-seeking process, it does not develop spon- 
taneously. Its development and change are influenced by individual and contextual factors (Morris et al., 2012). 
Therefore, it is necessary to examine students’ scientific reasoning skills and to elaborate activities that support the 
development of reasoning from early childhood. This requires valid and reliable instruments. Despite the fact that 
the development of scientific reasoning in pre-school and primary school is a dynamic area of research (O'Connor 
et al., 2021), relatively few measurement tools are available. One of these is the Science-K Inventory (SK-l), which has 
been the most widely used measure among German children (see Koerber & Osterhaus, 2019; Osterhaus & Koerber, 
2023). Therefore, this research examines the usability of the Hungarian adaptation of the Science-K Inventory in 
both paper-based and computer-based testing environments. The paper-based survey is a modified version of 
the original survey procedure, while the development of the computer-based version represents a new innova- 
tion in the history of the Science-K Inventory because this test has not yet been applied in a digital environment. 


Literature Review 
Evaluating Early Scientific Reasoning 


Common methods of assessing young people’s scientific reasoning include experiments carried out inde- 
pendently with ordinary or simple hands-on physical apparatus, and problem-based and story-based simulations 
(Mayer et al., 2014). Students’ performance on the test is influenced by the type, complexity, and level of abstraction 
of the tasks used (Lazonder & Kamp, 2012). The difficulty and complexity of the components of basic scientific rea- 
soning can be effectively assessed using tasks that are embedded in problem-based stories, use few variables, are 
cognitively less demanding, and do not involve children’s prior emotional beliefs about the story (Schulz & Gopnik, 
2004). Research by Bullock and Ziegler (1999) showed that when identifying and controlling variables in tests of 
scientific reasoning, students performed better when they were given a choice of closed-response alternatives 
rather than having to formulate the correct answer themselves. In multiple-choice tests, students who performed 
well were able to explain their choice thus demonstrating their understanding of the controlled test (Mayer et al., 
2014). Paper and pencil tests, which have recently transitioned to online platforms for data collection based on 
practical tasks, offer advantages such as practicality and the ability to carry out large-scale measurements with 
larger sample sizes. They also facilitate the examination of relationships with other cognitive and affective factors 
(Strand-Cary & Klahr, 2008). 


The Science-K Inventory 


The above criteria are well met by the Science-K Inventory developed by Koerber and Osterhaus (2019), 
which is designed to comprehensively measure scientific reasoning skills in pre-schoolers. The Science-K inventory 
comprises 30 multiple-choice items with a brief story and is divided into three components, namely experimenta- 
tion, data interpretation, and understanding the nature of science. Each component consists of 10 items. In the 
experimentation dimension, students are asked to choose the most appropriate experiment to test the research 
problem being investigated and to solve tasks that require them to correctly apply the CVS to test hypotheses. CVS 
is a crucial indicator of the development of scientific reasoning (Schwichow et al., 2020). In the data interpretation 
component, children are asked to identify covariation patterns (i.e., perfect, imperfect or non-covariations) in the 
data and then use the identified patterns to formulate a hypothesis. They also need to understand that when they 
encounter confusing or contradictory data, it will be challenging to draw clear conclusions. The third part of the 
exercises focuses on understanding the nature of science. The tasks are designed to address questions such as 
what scientists do as part of their job and what research questions they ask. 

The SK-I was validated in German (Koerber & Osterhaus, 2019) and has also demonstrated validity and reli- 
ability in other languages and cultures, such as Chinese (Osterhaus et al., 2023). Despite being primarily designed 
for kindergarten children, the SK-I has also been tested with third graders, where it performed well (Nyberg et al., 


1051 


https://doi.org/10.33225/jbse/23.22.1050 LQ 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


MEASURING SCIENTIFIC REASONING OF FOURTH GRADERS: VALIDATION OF THE SCIENGE-K ISSN 1648-3898 /Print/ 
INVENTORY IN PAPER-BASED AND COMPUTER-BASED TESTING ENVIRONMENTS 
(pP.1050-1062) ISSN 2538-7 1 38 /Online/ 


2020), and has been used in a longitudinal study of children from five to ten years of age (Osterhaus & Koerber, 
2023). The data collection method used in previous studies was paper-based, utilizing interviews (where the items 
were presented in an illustrated booklet) for kindergarten children and group tests (read out by an experimenter 
and presented in a PowerPoint presentation) for primary school children (Koerber & Osterhaus, 2019; Nyberg et 
al., 2020; Osterhaus et al., 2023; Osterhaus & Koerber, 2023). 


Computer-based Assessment of Scientific Reasoning 


Computer-based assessment has now almost completely taken over the assessment of the learning process, 
surpassing the use of paper-based tests. Rapid advances in technology have made it possible to assess more com- 
plex skills (e.g., critical thinking (e.g., Rosen & Tager, 2014), problem-solving (e.g., Wu & Molnar, 2018), creativity (e.g., 
Pasztor et al., 2015) and collaboration (e.g., Rosen, 2015) more reliably through computer-based testing (Shute & 
Rahimi, 2017). Another advantage is that multimedia elements, simulations, and dynamic items can be integrated 
into the testing environment (Csap6 et al., 2014). Furthermore, computer-based testing in large-scale assessment 
programs using item banks allows for targeted and adaptive test development and creation. This form of testing 
reduces the resources needed to carry out tests. Additionally, it enables rapid communication of test results and 
personalized evaluator feedback (Bennet, 2003; Quellmalz & Pellegrino, 2009). Recognizing these benefits, one 
of the most comprehensive international assessments, the OECD’s PISA (Programme for International Student 
Assessment) test, also completely switched (except for nine countries) to a computer-based testing environment 
in 2018 (OECD, 2019). 

There is a substantial body of research in the literature on the validation of tests in a digital environment re- 
garding scientific reasoning and inquiry skills. The main research questions of these research directions are whether 
paper-based and computer-based tests are equivalent in terms of internal consistency and validity (Williamson et 
al., 2017). For example, Vo and Csap6 (2023) conducted a study examining the performance of 731 students on the 
application of the control-of-variables strategy in basic physics. The study compared paper-based testing under 
teacher supervision with online testing without teacher supervision. The findings indicated that paper-based test- 
ing exhibited better reliability and total scores compared to online testing. However, the validity of the two tests 
was found to be equivalent regardless of the difference in delivery modalities (Vo & Csap6, 2023). Another study 
(Schroeders et al., 2013) found that the test medium (paper-pencil or technology-based) is not a determinant of 
comprehension ability in the natural sciences. Research conducted by Hallddérsson and colleagues (2009) examines 
modality effects on gender performance by comparing the scientific literacy results in the PISA 2006 assessment, 
both in paper-and-pencil format and computer-based format. The findings indicate that in all three participating 
countries (Iceland, Denmark, and Korea), boys outperformed girls in science literacy when the test was adminis- 
tered via computer. However, there are studies that have shown no gender differences in student performance in 
science tests in either test setting (e.g., Adanir et al., 2020; Brallier et al., 2015) In addition, it is important to note 
that learners tend to prefer computer-based testing, which can impact their test-taking attitudes and self-efficacy 
(Nikou & Economides, 2016; OECD, 2010). 


Research Aim and Research Questions 


The aim of the present study has been to explore the potential of paper-based and computer-based assess- 
ments for the Science-K Inventory among fourth graders in primary school. For this purpose, the SK-I was adapted 
to Hungarian, transferred into an online interface, and the paper-based (PB) and computer-based (CB) test versions 
were compared by addressing the following research questions: 

RQ1: Does the computer-based test measure similar reliability to the paper-based format? 

RQ2: Is there any evidence of equivalence between the online and paper-based groups on the SK-I at 
the item and task levels? 

RQ3: Is there a gender difference in students’ overall test scores and their scores across different test 
contexts? 

RQ4: How do students perceive the difficulty of the test, and how interesting did they find the tasks? 
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Research Methodology 
General Background 


This cross-sectional study employed a quantitative approach and utilized the Rasch model, based on item 
response theory (IRT), to illustrate the probability of an individual successfully solving an item in two test modes. 
This probability is determined by the relationship between the latent variable of ability and the item’s difficulty 
expressed on a linear scale (Rasch, 1960; Vo & Csap6, 2023). Internal consistency was measured using Cronbach's 
alpha to examine whether the reliability of the Science-K Inventory, which is employed for assessing the scientific 
reasoning of 4th-grade students in a computer-based testing environment, was consistent with that of the paper- 
based version. 

It is important to emphasize that the research described in this paper represents the outcome of the initial 
data collection phase (pilot study) conducted on a small sample after adapting the instrument. This phase served 
as the initial validation step in a broader research project aimed at large-scale assessment of the scientific reason- 
ing of primary school students within a digital environment, which is ending this year. 


Participants 


The pilot study involved 84 fourth-grade students (M40 10.18, SD = 0.47; 50.0% girls) from three classes in 
a primary school in Szeged. One of the classes participated in the Golden Gate English Playschool program (an 
accredited English language program), which is very popular among parents. As a result, in this particular class 
students of high ability and a good social background were overrepresented. To determine who would take paper- 
based tests and who would take computer-based tests, a random number generator was used to select students 
randomly for all three classes. Thus, 39 students (53.8% girls) completed the paper-based test, while 45 students 
(46.7% girls) completed the computer-based test. The absence of some students at the time of testing resulted in 
a slight imbalance in the proportions between the two test modes. At the beginning of the school year, the school 
administration asked parents for written permission for their children to participate in educational research. 


Paper- and Computer-based Instruments for the Assessment of Scientific Reasoning 


The students’ scientific reasoning was assessed using the Science-K Inventory which comprises 30 multiple- 
choice items divided into 3 components (10 items each): experimentation (EXP), data interpretation (DAT), and 
understanding the nature of science (NOS) (Koerber & Osterhaus, 2019). For each item, children are required to 
choose the best out of three answer options. Tasks of varying difficulty are adapted to children’s emerging scientific 
reasoning processes. For the purposes of scoring, each correct answer was worth 1 point, allowing students to 
obtain a maximum score of 30 points for answering all 30 items correctly. The test makers emphasized that the test 
is not a tool for measuring sub-skills, but rather an instrument for providing a more comprehensive and general 
assessment of scientific reasoning (Koerber & Osterhaus, 2019). For this reason, we considered the test as having 
a one-dimensional structure when analysing and evaluating the data. 

The adaptation procedure, which involved the contribution of Christopher Osterhaus, one of the developers of 
the instrument, was carried out in accordance with the established rules. Assistance was received from an English- 
speaking translator and an independent expert, both of whom provided support in the research undertaken. In 
contrast to previous group tests, where data was collected with the assistance of an experimenter who read the 
tasks aloud and the students had to mark the correct answer ona score sheet, in this study, students were asked to 
complete the test individually. In adapting the instrument, available slides were utilized to generate a paper-based 
test booklet and its digital version. Throughout the process, efforts were made to preserve the original structure 
and illustrations of the instrument. The only difference between the paper-based and digital versions of the test 
we created is the method of administration: in the paper-based test, the letter of the correct answer was written 
on the dotted line or encircled; in the digital version, the student had to click on the image of the correct answer 
in the experimentation and NOS subtests, while in the data interpretation subtest, the student had to click an icon 
to the left of the correct answer. Examples of these implementations are shown in Figures 1 and 2. 
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Figure 1 
Sample Item from the Paper-based (a) and the Computer-based (b) Version of an Experimentation Task 


(a) Tomi azt akarja kideriteni, hogy Maja j6-e kirakézasban. 
Mit kell Majanak tennie? 


Majanak ki kell raknia a Majanak egy néhany darabbol | | Majanak egy sok darabbol allo 
kedvenc kirakosat. allo kirakost kell kiraknia. kirakost kell kiraknia. 


ird a helyes valasz betijelét a pontozott vonalra! ..... 


b 
( ) Tomi azt akarja kideriteni, hogy Maja jo-e kirakozasban. Mit kell Majanak tennie? 
Kattints a megfeleld képre! 


Majanak ki kell raknia a Majanak egy néhany darabbol | | Majanak egy sok darabbol allo 
kedvenc kirakosat. allo Kirakost kell kiraknia. kirakost kell kiraknia. 


© Vissza Tovabb © 


Note. Tom wants to find out if Mia is good at doing puzzles. What does she have to do? Write the letter of the correct answer on 
the dotted line/Click on the correct picture. 
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Figure 2 


Sample Item from the Paper-based (a) and the Computer-based (b) Version of a Data Interpretation Task 


(a) 


—_——_—_—~ 
Robi ugy véli, hogy a zéld ragotol kihullanak ae 


se a 
a gyerekek fogai. <= ep _ 6 A 
PAY \ ~ & 


Most Robi megnézi a kéepeket a zéld és a piros ragogumikrol! Tee fi 
oe e a 
Ezek a gyerekek piros ragogumit ‘wW, J ws 
ragtak, és kihullott a foguk. EE ae 
Ezek a gyerekek Z6ld ragogumit -s o @e 
ragtak, és mindegyikiiknek &) th & 


egeszségesek a fogai. 


Robi kezdetben ugy gondolta, hogy a zdld rag6gumi miatt hullanak ki a gyerekek fogai. Vajon mit gondol 
most Robi, miutan megnézte a képeket? Karikdzd be a helyes vdlasz bettijelét! 


A) Azt gondolja, hogy a z6ld rag6 miatt hullanak ki a gyerekek fogai. 
B) Azt gondolja, hogy a piros rag6 miatt hullanak ki a gyerekek fogai. 
C) Azt gondolja, hogy nem szamit, hogy zdld vagy piros rag6gumit ragnak a gyerekek. 


Robi Ugy véli, hogy a zéld ragotol kihullanak sam 
a gyerekek fogai. ov 
‘ed 


Most Robi megnézi a képeket a Zdld és a piros ragogumikrol! 


&: 
Ezek a gyerekek piros ragogumit ee) 


ragtak, és kihullott a foguk. 


(b) 


Ezek a gyerekek zdld ragdgumit 
ragtak, és mindegyikiiknek 
egészségesek a fogai. 


E) 

E&) 

@5) 

&) 
oe @* 


Robi kezdetben lgy gondolta, hogy a zéld ragégumi miatt hullanak ki a gyerekek fogai. Vajon mit 
gondol most Robi, miutan megnézte a képeket? Kattintassal valaszolj! 


©  Azt gondolja, hogy a zdld ragogumi miatt hullanak ki a gyerekek fogai. 
°  Azt gondolja, hogy a piros ragogumi miatt hullanak ki a gyerekek fogai. 
Azt gondolja, hogy nem szamit, hogy zéld vagy piros ragogumit ragnak 


a gyerekek. 
© Vissza Tovabb © 


Note. He initially thought that green chewing gum causes teeth to fall out. What does Robby believe now? Circle the correct 
answer/Click on the correct answer. 


Measuring Test-Takers’ Perceptions 
To obtain students’ views, at the end of the tests, students were asked to indicate how difficult they found 


the tasks and how much they liked them. The items were rated on a four-point scale (1 = not at all, 2 = not very, 3 
= quite a bit, 4 = completely). 
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Procedure 


The paper-based and computer-based data collection took place simultaneously at the school in May 2023 
and lasted approximately 20-30 minutes. The paper-based test was administered with a colour test booklet, while 
the online data collection was conducted in the school’s computer labs using the eDIA (Electronic Diagnostic As- 
sessment) system (Csap6 & Molnar, 2019). 


Data Analysis 


Jamovi (version 2.3.26) was utilized for conducting the Rasch model based on item response theory (IRT) and 
differential item functioning (DIF) analysis. The aim was to examine whether there was a difference between the 
two test modes at the item level. SPSS (version 27) was employed for conducting descriptive statistical analysis, 
such as calculating means, standard deviations and percentage, and also for calculating Cronbach's alphas to 
assess the reliability of the two test modes. Additionally, SPSS was used to perform t-tests to analyse differences 
between genders. 


Research Results 
Reliability and Validity 


To test the reliability of the SK-I, Cronbach's alpha (a) values were computed. Alpha values were found to be 
similar for both the paper-based (a = .65) and computer-based (a = .70) tests. However, two items did not exhibit a 
good fit in the reliability analysis. Consequently, considering the item-rest correlations (Rin), it was deemed necessary 
to remove items DAT2 (Rir,.= -.14, Rir,,= .00) and DAT9 (Rir,, = .04, Rir.,= .00) from the analysis. After the necessary 
corrections — the possible causes of which are discussed in detail in the discussion section — the Cronbach's alphas 
with the 28 items were as follows: O,, = -71, O., = .73. These values are found to be acceptable. 

Table 1 summarizes the psychometric properties of the SK-I comparing the paper-based and computer-based 
groups. Considering the percentage of correct answers and item difficulty, it was observed that item 16 (DAT6) 
proved to be the most challenging in both the paper-based (diff. 0.47) and computer-based (diff. 0.57) tests. This 
particular item assessed the interpretation of confounded data. However, the results suggested that overall the 
test was easy for students to complete (all except item 16 have negative values), irrespective of the medium. In 
this case, the ceiling effect was not a problem, as the test was used to measure the level of students’ scientific 
reasoning by the end of Grade 4. 

The Rasch model analysis indicated that the items fit the model well for both test modes. In the paper-based 
test, the infit values (weighted mean squares, MNSQ) ranged from 0.86 to 1.18 (M=0.99, SD = 0.10), whereas in the 
digital test, they ranged from 0.75 to 1.34 (M = 0.99, SD = 0.12). To address the question of whether there are any 
items with unexpected behaviour in the test, a differential item functioning (DIF) analysis was conducted using 
binary logistic regression in dichotomous items. The DIF analysis revealed no significant difference in the behaviour 
of any of the items between the two test modes. The same analysis method applied to gender did not identify any 
items exhibiting unexpected behaviour in any of the test modes. 


Table 1 
The Psychometric Parameters of the SK-I by Mode of Administration 


Correct answer Difficulty Infit 
No. Item 
Paper-based Computer-based Paper-based Computer-based Paper-based Computer-based 
1 EXP‘ 82.86 80.00 -1.79 -1.55 1.03 0.88 
2 EXP2 85.71 62.22 -2.02 -0.56 0.92 0.92 
3 EXP3 85.71 84.44 -2.02 -1.89 0.91 0.86 
4 EXP4 62.86 53.33 -0.61 -0.14 0.94 0.93 
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Correct answer 


Paper-based Computer-based 


No. Item 
5 EXP5 82.86 
6 EXP6 97.14 
7 EXP7 97.14 
8 EXP8 97.14 
9 EXP9 97.14 
10 EXP10 94.29 
11 DAT1 94.29 
12 DAT2 51.43 
13 DAT3 85.71 
14 DAT4 51.43 
15 DAT5 60.00 
16 DAT6 40.00 
17 DAT? 62.86 
18 DAT8 77.14 
19 DAT9 56.41 
20 DAT10 85.71 
21 NOS1 74.29 
22 NOS2 85.71 
23 NOS3 65.71 
24 NOS4 97.14 
25 NOS5 91.43 
26 NOS6 60.00 
27 NOS7 82.86 
28 NOS8 91.43 
29 NOS9 82.86 
30 NOS10 85.71 


Descriptive Comparison of the Students’ Performance 


86.67 
93.33 
97.78 
93.33 
97.78 
91.11 
84.44 
35.62 
84.44 
53.33 
46.67 
37.78 
62.22 
86.67 
40.02 
84.44 
55.56 
82.22 
57.78 
97.78 
82.22 
57.78 
66.67 
93.33 
88.89 
91.11 


Computer-based Paper-based 


Difficulty 
Paper-based 
-1.79 -2.09 
-3.85 -2.91 
-3.85 -4.09 
-3.85 -2.91 
-3.85 -4.09 
-3.10 -2.58 
-3.10 -1.89 
-2.02 -1.89 
-0.07 -0.14 
-0.47 0.16 
0.47 0.57 
-0.61 -0.56 
-1.39 -2.09 
-2.02 -1.89 
-1.21 -0.24 
-2.02 -1.71 
-0.75 -0.35 
-3.85 -4.09 
-2.64 1.71 
-0.47 -0.35 
-1.79 -0.78 
-2.64 -2.91 
-1.79 -2.31 
-2.02 -2.58 


1.14 
1.07 


0.89 


Infit 


Computer-based 


0.96 
1.08 
1.07 
0.86 
0.96 


The average performance of students on the paper-based test was 80.7% (SD = 12.4%) and on the computer- 
based test 77.1% (SD = 13.3%). An independent samples t-test showed that there was no significant difference 
(t(82) = 1.38, p =.172) between the student's performance in the two media. Gender differences were examined by 
comparing the results of girls and boys using an independent samples t-test. The results are being presented in Table 
2. It can be said that there was no significant difference between the performance of boys and girls in the paper- 
based (t(37) = 0.14, p = .889) or computer-based (t(43) = 0.69, p = .493) testing modes. Furthermore, there was no 
significant difference in test modality between the boys '(t(40) = 0.57, p =.573) and girls’ (t(40) = 1.39, p=.172) groups. 
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Comparison of Student Performance on SK-I by Gender and Test Modality 


Paper-based (% correct) Computer-based (% correct) 


Group t p Cohen’s d 
n M (SD) n M (SD) 

All 39 80.7 (12.4) 45 77.1 (13.3) 1.38 172 0.30 
Boys 18 80.7 (12.3) 24 78.2 (13.3) 0.57 573 0.17 
Girls 21 81.1 (12.7) 21 75.4 (13.5) 1.39 172 0.43 

t 0.14 0.69 
p 889 493 
Cohen's d 0.05 0.21 


Test-Taker Perceptions 


Table 3 displays the perceptions of paper-based and computer-based test-takers. The low mean scores 
(M,, = 1.32, SD = 0.54; M_., = 1.55, SD = 0,66) for test difficulty indicated that students found the test easy in both 
modes, which was consistent with the results of item difficulty. Consequently, it can be concluded that the fourth 
graders made a realistic assessment of the test's difficulty. The high average scores of test enjoyment (M,, = 3.53, 
SD = 0.66; M_, = 3.47, SD = 0.63) indicated that both paper and computer-based respondents found these types 
of tasks interesting and enjoyable to solve. When comparing test modes, no significant differences were found in 
either test difficulty (t(82) = 1.59, p = .116) or enjoyment of the test (t(82) = 0.43, p = .668). 


Table 3 
Students’ Perceptions Across Modes 


Paper-based Computer-based 
(n = 39) (n= 45) 
Item t p Cohen’s d 
M (SD) M (SD) 
Test difficulty 1.32 (0.54) 1.55 (0.66) 1.59 116 0.36 
Test enjoyment 3.53 (0.66) 3.47 (0.63) 0.43 668 0.09 


Discussion 


A key concern in transitioning from a paper-based test to a digital format is always whether the measurement 
instrument utilized in the new testing environment can maintain similar validity to its paper-based counterpart (Wil- 
liamson et al., 2017). Consequently, the majority of research questions (RQ1-RQ3) were focused on addressing this 
matter. In assessing the validity of the measurement, we discovered that the reliability of the SK-I was acceptable for 
both paper-based and computer-based data collection. The Cronbach's alpha values were nearly identical to those 
of the instrument developed for the German sample in the same grade during group testing (Osterhaus & Koerber, 
2023). This result may validate the efficacy of this data recording method for the SK-I, confirming the hypothesis that 
by grade 4, students’ text comprehension skills have developed to the point where they can independently complete 
reasoning skills tests. 

The two items (DAT2 and DAT9) that exhibited low item-rest correlations measured the interpretation of covaria- 
tion data, including data of imperfect type. It can be observed that in both the computer- and paper-based versions, 
these two items were solved with the lowest accuracy by the participants, except for item DAT6 (see Table 1). It can 
be assumed that, in grade 4, students are not yet proficient in separating the effects of covariation data for different 
levels of complexity. Therefore, even though they solved the complete test with high scores, these two items (DAT2 
and DAT9) proved to be too difficult for them to interpret. The German fourth-graders also made the highest number 
of mistakes on these two items (Osterhaus & Koerber, 2023), leading to unexpected behaviour and not fitting the 
pattern of their performance in the complete test. Consequently, this situation impacted the test's reliability. 
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The internal validity of the SK-I, assessed through DIF analysis, confirmed that the two methods of data collection 
are comparably acceptable. As a result, no item-level differences in media effects were observed between the two 
testing methods. The reliability and validity of both the paper-based and computer-based test forms were found to 
be equivalent. Previous studies have also yielded similar results, in which paper-based tests of reasoning skills were 
converted to a digital format (e.g., Hassler Hallstedt & Ghaderi, 2018; Molnar et al., 2011; Vo & Csapo, 2023). 

The sample exhibited a ceiling effect (Staus et al., 2021), with a significant proportion of students attaining high 
test scores. Nevertheless, this does not pose a problem for the utilization of the test, as the primary objective of the 
SK-l is to monitor the progression of scientific reasoning. Hence, the high scores indicate that the fourth graders in the 
study have already reached the expected level of scientific reasoning development for their age. The similarly high 
scores (75.9%) of German students in the same age group are interesting (Osterhaus & Koerber, 2023). 

Before the PISA test was transitioned to an online format, a series of pilot studies were conducted to examine 
whether there were any differences in student performance when the test medium was altered (e.g., Bjornsson, 
2008; Jerrim et al., 2018; Kroehne et al., 2019). Jerrim’s (2016) study found that, despite a strong correlation between 
paper-based and digital versions of the math test, significant differences exist in the results. However, it is important 
to note that while the assessment framework remained the same, certain test items incorporated specific task types 
that exploited the opportunities provided by digital testing (OECD, 2014). The results showed no significant difference 
in mean performance when the test medium was switched, indicating that the digital form of the test is suitable for 
assessing the scientific reasoning of fourth-grade students. Another study, analysing PISA 2006 data, reported that 
by changing the test medium, boys outperformed girls in scientific literacy in a manner that was not observed in 
the paper-and-pencil test. This phenomenon was attributed to boys having a lower reading load and experiencing 
greater test fatigue on low difficulty items in paper-based tests (Halldérsson et al., 2009). However, none of these 
phenomena were observed. Changing the test medium did not have a significant impact on test performance. This 
aligns with the initial assumption, as the only change made was in the medium, without altering the test's presenta- 
tion or structure. While the influence of paper versus digital media on reading comprehension could have affected 
these results (Delgado et al., 2018), this was not observed in this sample. Furthermore, with the conversion of the 
test recording mode to digital, the boy’s performance in the computer-based test did not surpass that of the girls. 

Gender is a crucial variable in assessing scientific reasoning, so one of the research questions concerned gender 
differences. Several studies confirm that boys tend to perform better on scientific reasoning ability tests (see Ha et 
al., 2021; Lazonder et al., 2020; Luo et al., 2021), while other studies find no significant differences in reasoning abil- 
ity (see Koerber et al., 2015; Mayer et al., 2014; Molnar, 2011; Osterhaus & Koerber, 2023). The results of this research 
support the latter perspective, as we did not find any significant differences in boys’ and girls’ scientific reasoning 
across any of the test types. 

Some previous studies have indicated that students generally prefer computer-based testing over paper-based 
testing (e.g., Okocha, 2022; Tella & Bashorun, 2011). Therefore, one of the research questions (RQ4) aimed to examine 
whether there are differences in students’ perceptions of test difficulty and test enjoyment as a function of the test 
medium. The results suggest that the test medium did not have a significant effect on test takers’ perceptions. 


Conclusions and Implications 


The main aim of this research was to demonstrate that the Hungarian version of the Science-K Inventory can 
be effectively utilized to evaluate the scientific reasoning (experimentation, data interpretation, understanding the 
nature of science) of fourth-grade students, not only in a traditional paper-based setting but also in a computer-based 
testing environment. The novelty of this research was that the SK-I had not yet been tested in a computer-based 
format. Previously, data collection for third and fourth graders was conducted in a paper-based group setting with 
test assistants who read out the instructions for the visually presented tasks. It was therefore questionable whether 
unassisted computer-based data collection could be employed in this particular age group. Based on these results, it 
can be concluded that the test-taking method used is suitable for fourth graders and that both the paper-based and 
digital-based tests exhibit comparable reliability and validity indicators. As a result, both modes are equally valid for 
assessing fourth graders’ scientific reasoning. 

The study carries several implications. The development of scientific reasoning and research skills is crucial in 
the teaching and learning of science. However, there is a lack of reliable measures available to assess young children’s 
understanding of scientific inquiry and scientific reasoning skills. Therefore, the introduction of reliable measurement 
tools such as the SK-I into teaching practice that allow the monitoring of changes in students’ reasoning and individual 
differences in this area is valuable. 
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Computer-based testing offers several advantages that can be exploited, such as the facilitation of large-sample 
testing by reducing resource requirements. It can also provide rapid and valuable information for teachers to aid the 
planning of the development of scientific literacy competence. Furthermore, the research results can contribute to the 
design of programs aimed at improving scientific literacy and serve as an early diagnostic tool to better understand 
the decline in students’ performance in international science assessments, such as PISA (Programme for International 
Student Assessment) and TIMSS (Trends in International Mathematics and Science Study). 

A limitation of this study is the small sample size, which limits the ability to obtain more robust item parameter 
estimates for Rasch analysis, and to test the construct validity of SK-I by confirmatory factor analysis (CFA). However, 
future research aims to carry out a study with an expanded sample size to acquire more precise model fit statistics for 
the Rasch model and CFA, along with more accurate item parameters. Also, it may be worth investigating whether 
the observed item fit problems persist in the case of a larger sample. If so, exploring the contextual issues or curricular 
and cultural differences that give rise to these problems could indicate a need for revisions to the Hungarian version 
of the SK-I. Additionally, there are plans to test SK-I with younger children, which, when narrated by voice, could be 
used to assess the scientific reasoning of first and second graders and thus conduct a longitudinal study. Exploration 
of the relationship between scientific reasoning and additional variables, including school performance, attitudes 
towards school subjects, motivation to learn science, and social background, is also part of the next research agenda. 
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LESSONS: AN EXPERIENCE 
SAMPLING APPROACH 
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Elisa Vilhunen, 
Jari Lavonen, 
Kaido Reivelt 


Introduction 


The question “How to make science more interesting and engaging to 
students?” has challenged both teachers and policymakers for over a hundred 
years. According to Roberts and Bybee (2014), science education aims to 
equip students with the knowledge, skills, and attitudes necessary to un- 
derstand the world around us, to ask relevant questions and to find answers 
through demonstrations, planned investigations, or through the search of 
information. Achieving this aim can be supported by having students engage 
in scientific practices. Scientific practices are similar to expert performance 
in the discipline. They involve asking questions, planning and carrying out 
investigations, analysing and interpreting data, developing explanations and 
building models based on the data. Scientific practices are not the same as 
inquiry, nor do they replace inquiry. Rather, they consist of a combination of 
activities in teaching and learning situations (Krajick & Merritt, 2012). 

Students experience school science subjects, especially physics, as a 
complex and tedious subject (Blajvaz et al., 2022; Fidan & Tuncel, 2021). How 
students perceive the subject influences their motivation to study it, which 
in turn affects the effort the students are willing to make for that subject 
(Debacker & Nelson, 2000). Thus, it is no surprise that a poor experience in 
physics as a subject leads to lower-than-expected academic achievement 
(Barmby & Defty, 2006). To overcome this issue, researchers and teachers 
have been looking for new ways of teaching physics to make learning of 
physics interesting to students. Rotgans and Schmidt (2011) found in their 
study that situational interest predicts the active engagement of students, 
which in turn predicts academic achievement. This implies that interest is 
a crucial aspect to focus on when making efforts to improve the quality of 
science education and increasing scientific literacy (Lamanauskas, 2022). 
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Abstract. Enhancing interest in 

physics learning has been an important 
component of education policy and policy 
implementation for decades. However, in 
many countries, student interest in physics 
has not increased. The aim of this study 
was to predict situational interest using 
individual interest and the instructional 
activities students engaged in. Also, 

the suitability of the data collection 
approach to measure situational interest 
was assessed. A teaching module about 
real-world oscillations was implemented 
to 179 Estonian lower secondary school 
students. Data about situational interest 
were collected using the experience 
sampling method (ESM), which was 
measured three times during the module. 
The results indicated that gender, grade, 
and instructional activities did not predict 
situational interest. Individual interest was 
a significant predictor and correlation with 
situational interest increased with time, 
contrary to previous findings. The ESM 
approach used in this study was considered 
to be effective in disturbing students’ study 
flow minimally but may have affected 

the amount of missing data. This study 
contributes to the research done on student 
interest in a classroom setting by indicating 
amore complex relationship between 
situational and individual interest than 
previously suggested. 
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Interest is a multi-faceted concept described by Hidi and Renninger (2006) as a “psychological state of en- 
gaging or the predisposition to re-engage with particular classes of objects, events, or ideas over time” (p. 112) 
Interest is divided into situational and individual interest (Krapp et al., 1992) and according to the four-phase 
model of interest development by Hidi and Renninger (2006), situational interest is composed of triggered and 
maintained interest phases; individual interest is composed of emerging and well-developed interest phases. Trig- 
gered situational interest can be initiated for example by information that is surprising or personally relevant and 
it is usually supported by an external factor. Maintained situational interest is similar in the way that it is externally 
supported, but the maintaining is achieved through personal involvement in the act that triggered the interest. 
This phase can lead to the development of individual interest, but it is not always so. The next phase, emerging 
individual interest, is typically self-generated and doesn't need external support to exist, but it does help in moving 
to the final stage of well-developed individual interest. This final phase is characterised by perseverance in tasks 
even in the face of frustration and requires the least amount of external support. It is possible to move backward, 
i.e, maintain interest in something for a time but then lose interest (Hidi & Baird, 1988; Hoffmann, 2002; Krapp & 
Prenzel, 2011; Renninger et al., 2019). 

In the context of science education, the measures of situational interest are more valuable than individual 
interest because situational interest is more dynamic and susceptible to external factors. Bahtaji (2023) has found 
that interest developed during a lesson has a stronger impact on understanding science concepts than the inter- 
est developed prior to the lesson. This is also supported by Rotgans and Schmidt (2018), who found that a lack of 
individual interest can be compensated by a well-presented problem and that only situational interest predicted 
task performance, not prior individual interest. This provides opportunities to gauge how different teaching ap- 
proaches (environment, methods, teachers) affect students’ interest, and thus indirectly their academic achievement. 
Based on such data, necessary curricula or teacher training changes can be developed to improve student learning. 

Although the four-phase model of interest explains well how interest develops, it does not encompass all 
aspects. Based on his well-known studies on situational interest Haussler (1987) has identified three dimensions 
present in the construct: interest in a particular topic, interest in the context and interest in the activity engaged in 
in tandem with the topic and context. For example, some students may find a lesson about hydrodynamics fasci- 
nating because it is crucial to the way houses are supplied with water (interest in the context), but care little about 
the theoretical foundations or the calculations accompanying it (interest in the topic). Interest in the instructional 
activity engaged in is the focus of this study — how instructional activities associated with scientific practices influ- 
ence students’ situational interest. Instructional activities are activities by which students obtain new knowledge 
or reinforce already obtained knowledge in lessons; examples include listening, experimenting, and completing 
worksheets. The term “instructional” does not refer to a teacher-centred approach, it emphasises that teachers plan 
and manage the activities students engage in. The role of teachers in instructional activities is to guide students 
to make sense of the phenomena. Scientific practices are essentially the activities that scientists engage in during 
their everyday work. Such activities include, for example, planning and carrying out investigations, analysing data, 
interpreting, and asking questions. 


Measuring Situational Interest 


Measuring interest is a difficult task as there are many aspects to consider when designing such measurement: 
the data collection method in general; the scale and frequency of measurements; the number of items in a ques- 
tionnaire and the factors influencing interest. The experience sampling method has gained popularity in measuring 
situational interest because it is very flexible, enabling the collection of data very close to learning situations (Zirkel 
et al., 2015). The experience sampling method or ESM is a data collection method in which the study participants 
provide real-time self-reports about their experiences in a situation. These experiences include reports about 
experienced activities, thoughts and feelings. ESM typically uses short questionnaires, which are presented to the 
participants at random or semi-random times during the study (Hektner et al., 2007). When measuring situational 
interest or similar constructs, a four or 5-point Likert scale is typically used, and the number scale has sometimes 
been replaced with faces (Tapola et al., 2013). However, there is no clear consensus on the proper way to conduct 
an ESM study (Dejonckheere & Erbas, 2021), which is why, more research is required around its different aspects. 

It is known that the frequency of asking questions from the study participants and the number of items 
used in every questionnaire also play a vital role. Eisele et al. (2020) have studied both effects and have found 
that an increase in item count has a negative effect on response rate and data quality, in addition to increasing 
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the burden on students. Surprisingly, the frequency of questions didn’t have such negative effects. Lavonen et al. 
(2021) used ESM to ask students about situational interest three times during a 90-minute lesson and reported 
high response rates (over 80%). A suitable frequency depends on the construct of the study and how fast it can 
change. Studies about the evolution of situational interest are rare (Rotgans & Schmidt, 2011) and it is unknown 
what is the optimal frequency of measuring situational interest. The factors influencing situational interest are 
possibly the most significant issue to tackle, as there are far too many of them to account for in any study. Some of 
them, such as the topic, context and the type of activity can be asked in the ESM questionnaire. Some of the fac- 
tors, such as gender, previous individual interest or knowledge, can be studied using background questionnaires. 
However, several factors, such as the influence of peers or even how well a participant slept, could be challenging 
to account for. This is why much care must be taken to limit the variety of potentially influential factors and narrow 
the study's focus if any reliable conclusions are to be made (Eisele et al., 2020). 

This study was conducted in Estonia, where teachers are expected to achieve the learning outcomes detailed 
in the national curriculum, but the way these are achieved is up to them. However, there are admission tests and 
exams at the end of lower secondary school, which are highly focused on factual knowledge and textbook-style 
problems. This means that making sense of phenomena is of low priority to the teachers because it is rarely re- 
quired in the tests. Many physics teachers in Estonia make a clear distinction between theoretical and practical 
lessons. In theoretical lessons, the teacher focuses on explaining phenomena or simply gives ready answers and 
shows how various physics models are used for making predictions. Afterwards, the students are guided in solving 
textbook problems. Despite the teachers often presenting demonstrations during the lessons, they do not guide 
their students to inquiry activities. In practical lessons, the focus is on producing a phenomenon, but discussing 
the phenomenon’ link to what was learned in theoretical lessons is lacking. 


Research Questions 


Experimenting in a physics classroom has previously been criticised as being ineffective in its current form of 
teaching (Abrahams & Millar, 2008; Abrahams & Reiss, 2012). This criticism is mainly due to an unsuitable approach 
by the teachers, who often focus on replicating a research design according to a manual instead of guiding the 
students to make sense of a scientific phenomenon through engagement in scientific practices. The aim of this 
research was to predict situational interest in a physics teaching module with a heavy emphasis on experiments. 
Furthermore, a goal was to assess the suitability of the chosen data collection approach. The research questions 
used in this study were the following: 


1. Do instructional activities, individual interest, gender, and grade predict situational interest? 
2. Whichattributes of the instrument and its implementation but also the characteristics of the participants 
and their environment influenced the response rate of situational interest questions? 


Research Methodology 
General Background 


This study used the experience sampling method to measure the situational interest of lower secondary 
students in a physics teaching module. Measures of situational interest were evaluated in terms of gender, grade, 
group size and the instructional activities students engaged in during the module. Furthermore, correlations be- 
tween individual and situational interest and between each measure of situational interest were analysed. Based 
on the findings, linear regression models were fitted to predict student situational interest. Issues regarding the 
measurement of situational interest using the chosen design are discussed. This quantitative study was conducted 
in Estonia in the spring of 2022. Participants included ten student groups from seven different schools all over the 
country. 


Design 


A teaching module about oscillations was designed specifically for this study. This module focused heavily on 
planning and conducting experiments to make sense of oscillations. Tasks in the module included measurements, 
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calculations, and interpretation of the collected data by answering multiple-choice questions. This module was 
taught in the form of a workshop, which consisted of two to three 45-minute lessons with breaks in between. The 
reasons behind choosing a workshop over a regular lesson were: a) this allowed to take measurements over a lon- 
ger period of time, thus giving valuable data on the evolution of situational interest; b) experiments together with 
a theoretical introduction and the following discussion, the importance of which was emphasised by Abrahams 
and Millar (2008), take more time than a regular 45-minute lesson. As the aim of this module was to provide an 
in-depth understanding of oscillations and their importance in everyday life, as well as allow time for discussion, 
it was deemed the best course of action. 

The structure of the module was in the format of blended learning (Hockly, 2018), i.e., the learning materials, 
as well as the study questionnaires, were in the form of online worksheets, but students conducted experiments 
using physical objects. Digital materials, such as videos, instructions, and explanations of concepts, were grouped 
in an online learning environment. The environment could be accessed via a computer or a hand-held device, and 
participants had to enter a unique code to see the lesson materials. 


Participants 


The sample of this study was composed of students who participated in a physics workshop programme run 
by the University of Tartu. Each year approximately 7-10 (depending on the number of applying schools) lower 
secondary schools are accepted for the programme and from each school approximately 15-30 (depending on 
the number of students interested in participating) students take part in the workshops. A total of 179 students, 
of which 48% were boys and 52% were girls, participated in the study. The students were from different schools 
located in different parts of the country and formed ten groups. For each group, the module was taught by the 
first author of this article to keep the teaching as uniform as possible. However, the groups were very different in 
terms of interest, previous experiences with physics and overall academic achievement. One of the groups included 
students for whom Estonian was not their mother tongue. As this study utilised a physics workshop programme 
run by the University of Tartu, some constraints were imposed. The groups were a mix of 8" and 9th-grade students 
because the programme was aimed at all lower secondary school students who learn physics. This provided an 
opportunity to evaluate whether there was a difference in situational interest between the grades. The programme 
was intended as an extra-curricular activity and was thus voluntary. Some schools, however, made it mandatory 
for all students to participate and most of the workshops took place during school hours. It was decided by these 
schools that the workshops should substitute regular physics lessons. Thus, students with both very high and low 
interest participated in the study. Considering the above, the sample was representative of the student population 
of the country. The students were asked for consent at the beginning of the module and every student had the 
opportunity to stop participating at any moment. The collected data cannot be linked to the participants. 


Instruments 


Two instruments were used in this study: a background questionnaire and a short (taking less than a min- 
ute to complete) questionnaire that was administered using the experience sampling method. The background 
questionnaire used in the study had questions about gender, grade, and statements about individual interest. 
The statements for measuring individual interest were selected from the “Students’ View of Science” category of 
the 2015 PISA study (OECD, 2017). In the PISA study, statements about both the value and enjoyment dimensions 
of individual interest were present, but in this study, five statements of the enjoyment dimension were chosen. 
The statements used were: “Usually | am having fun learning about physics topics-; “I like reading about physics: 
“lam happy when I’m dealing with physics. “I enjoy obtaining new knowledge in physics.’ and “I am interested 
in learning about physics” All of the statements were on a 5-point Likert scale ranging from “strongly disagree” to 
“strongly agree”. The statements were translated into Estonian and modified so that the emphasis was on phys- 
ics instead of science. The questionnaire was administered to the students at the beginning of the module and 
included information about the study. 

The second instrument of the study was the ESM questionnaire, composed of three questions: “How interesting 
was the activity at hand?” “What was the activity?” and “With whom were you doing the activity?” It is common to 
have only a few questions or items in this type of experience sampling method questionnaire (Eisele et al., 2021). An 
ESM questionnaire is used in real classroom situations, and a small number of questions do not influence students’ 
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learning. The first question was on a 5-point Likert scale ranging from “no interest” to “very high interest.’ The other 
two questions were multiple-choice. For instructional activities, the choices were: answering worksheet questions, 
listening, watching a video, experimenting, and not engaging in instructional activities. The group question had 
the following options: alone, in a pair, or in a group of at least three people. 

Currently, there is no consensus on how to determine the reliability of single-item measurements (Eisele et al., 
2021). However, the validity of such items can be assessed by studying the relationship to other related variables. 
In this study, individual interest, which was measured using a tested multi-item questionnaire, was related to situ- 
ational interest. Correlation analysis between the two constructs confirmed a positive and predictable correlation, 
which is in line with other theories about interest (Hidi & Renninger, 2006). Several other studies have also used 
single items to measure interest and similar concepts in the context of ESM (Beymer et al., 2020; Lavonen et al., 
2021; Pekrun et al., 2017; Vilhunen et al., 2022, 2021). 


Research Procedure 


Each module lasted for two or three 45-minute sessions, depending on the school. A general picture of the 
instructional activities engaged in the module can be seen in Table 1. Students were asked to fill in the background 
questionnaire and afterwards pay attention to their tablets for when the ESM questionnaires would appear. Next, 
the workshop continued in pairs, or in rare cases, alone or in a group of three, depending on the number of students. 
The three ESM questionnaires were separated by roughly equal amounts of time, but each student was given ques- 
tions at a slightly different time. This allowed the researchers to collect data about a wide range of activities and had 
minimal negative effect on the classroom workflow, the importance of which is emphasised by Sinatra et al. (2015). 
As the study was voluntary, however, the students had the opportunity to skip the questionnaire if they did not 
wish to participate. It was not possible to reopen the questionnaire after it was closed on purpose or by accident. 


Table 1 
General Picture of the Module 


Activity Time 
Introduction of the study and answering the background questionnaire 10 min 
Short lecture on oscillations and calculations about oscillations 10-15 min 
The first experiment (building a pendulum) and the first ESM questionnaire 20-25 min 
Break 10-30 min 
The second experiment (measuring using sensors) and the second ESM questionnaire 20-25 min 
Third experiment (achieving resonance with the pendulum) 20-25 min 
Break 10-30 min 
Fourth experiment (building a model skyscraper) and third ESM questionnaire* 30 min 
Discussion about experiments and resonance in everyday life, conclusion 15 min 


Note. For schools which had only two lessons, the third ESM questionnaire (noted by the asterisk) appeared at the end of the 
second lesson. 


Data Analysis 


The RStudio environment (R Core Team, 2022) was used to analyse the data. For individual interest a mean 
value was calculated based on the five questions; the internal consistency between the five questions was high 
(a= .876). Mann-Whitney U-tests were used to compare genders and grades. A one-way ANOVA and a post-hoc 
test were used to compare the instructional activities based on situational interest. Kendall Tau correlation analysis 
was used for determining the correlations between individual and situational interest, between group size and 
situational interest and between each measure of situational interest. Finally, based on the analyses, linear regres- 
sion models were used to predict situational interest with the relevant variables. The same analysis methods were 
also used to explain the response rates of the ESM questionnaires. 
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Notes Taken During the Data Collection 


Taking field notes during data collection can supplement the data with contextual information as well as help 
explain issues that are not visible from the collected data (Nespor, 2006; Phillippi & Lauderdale, 2018). During the 
teaching, the first author took short notes on the groups that he taught. General information about the school was 
written down for every group, but other notes were taken only when something unexpected took place. The notes 
were in free form, written on the computer after the teaching and most of these included only a few sentences. 
The notes were important in explaining why some students had not answered all of the questions. Below are a 
few examples of the notes taken: 


1. “Group 1 - the first attempt with the instrument worked as intended: there was time for three 45-minute 
lessons, but the students were visibly tired, which is why | decided to end the module 15 minutes earlier. The 
students were moderately compliant with the instructions, but less so when they had to build the pendulums. 
For some students the internet browser closed unexpectedly, disrupting the data collection. The second ESM 
questionnaire appeared during the break, which lessened the number of useful answers obtained from the 
students” 

1. Collective notes about the workshops: “One big issue was the timing of the questionnaires. Since every 
school had breaks and lessons of different lengths and | learned this as after arriving at the school, there was 
nothing | could do to remedy the issue that at least one questionnaire appeared during a break. Another an- 
noyance were students who downloaded games on the tablets and largely ignored me for the remainder of 
the module. Even if| got some of them to answer the questions on the worksheets there was a small chance 
that they answered the ESM questionnaires as they were easy to dismiss.” 


Research Results 


There were no significant gender or grade differences in situational interest or individual interest, but there 
was a significant difference in the two individual interest questions. The questions were “/ am happy when | am 
dealing with physics” (p < .001) and “Usually | am having fun when learning about physics topics” (p < .05). For these 
questions, boys reported a higher level of interest than girls. Furthermore, levels of situational interest were higher 
than individual interest, and significant differences (p < .01) between individual interest and the second and third 
measures of situational interest were measured. 


Table 2 
Kendall Tau Correlation Coefficients between Different Measures of Interest 


Situational interest 1 Situational interest 2 Situational interest 3 
Individual interest 0.29 0.35 0.41 
Situational interest 1 - 0.57 0.29 
Situational interest 2 - - 0.46 


Note. The numbers behind situational interest indicate each separate measure and its order. The dependent variables are dis- 
played horizontally and the predicting variables vertically. 


There were significant (p < .001) correlations between individual interest and all measures of situational inter- 
est; correlations were also significant (p < .001) between the individual measures of situational interest (see Table 
2). The correlation between individual and situational interest increased as the module progressed. The strongest 
correlations were between the closest measures of situational interest. There was no significant correlation between 
group size and situational interest. One-way ANOVA analysis followed by a post-hoc test revealed that the only 
significant difference in situational interest was between the “experimenting” and “not engaged in instructional 
activities” options, with experimenting being considered more interesting. 95% confidence intervals overlapped 
for all other instructional activities (see Table 3). 
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Table 3 


PREDICTING SITUATIONAL INTEREST BY INDIVIDUAL INTEREST AND INSTRUCTIONAL 


ACTIVITIES IN PHYSICS LESSONS: AN EXPERIENCE SAMPLING APPROACH 
(PpP.1063-1073) 


Activities by Numbers, Their Corresponding Mean Situational Interest Levels and 95% Confidence Intervals 


Instructional activities 
Experimenting 
Listening 
Not engaged in instructional activities 
Watching a video 


Worksheet questions 


Mean situational interest 


3.73 


3.51 


2.6 


3.51 


3.45 


95% Cl 


3.56 - 3.91 


3.28 - 3.74 


1.88 - 3.32 


3.21 - 3.82 


2.99 — 3.91 


Based on these results, linear regression models were fitted to predict each measure of situational interest. 
Mixed-effect models were not used as intraclass correlations were negligible. Independent variables included 
individual interest and situational interest, where applicable; other variables were discarded. To determine the 
best fits, possible models were compared using the Akaike Information Criterion, (Table 4). 


Table 4 


Estimates for Predictors of Each Situational Interest Measure 


Predictors ee 

int. 1 

(Intercept) 1.84*** 

Individual int. 0.46*** 

Situational int. 1 

Situational int. 2 

Observations 124 

Marginal R2/ 0.122/ 

Conditional R2 0.114 


Note. *p < .05;**p < .01;***p < .001. 


95% Cl 


1.07 - 2.61 


0.24 - 0.69 


Situational 


int. 2 


0.79* 


0.27* 


0.55*** 


95% Cl 


0.08 - 1.50 


0.06 - 0.49 


0.39 - 0.71 


Situational 
int. 3 


0.57 


0.60*** 


0.31" 


103 


0.369/ 


0.356 


95% Cl 


-0.26 - 1.40 


0.34 - 0.86 


0.11 - 0.51 


The main issue encountered in this study was connected to the response rate. Before data analysis 17 students 
out of the initial 179 were excluded due to them not answering any or only one of the questions in the ESM ques- 
tionnaires. 11 other students were excluded due to inconsistent and/or missing answers. Nevertheless, a drop in 
response rates was clear when comparing the three ESM questionnaires. Compared to the first measure, response 
rates dropped by an average of 6% for the second and 14% for the third measure. Situational interest questions 
were answered the most, followed by the activity questions and then the group questions. Response rates for 
these were 90%, 84% and 77% for the first measure and 77%, 70% and 63% for the last measure, respectively. No 
significant differences in gender or grade were found for the response rates, neither was there any correlation with 
either individual or situational interest. There were some differences in groups: the non-native Estonian group had 
a significantly lower mean response rate compared to six other groups; the difference with the remaining three 
groups was not statistically significant. The average response rate for the background questions was 99%. 
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The results involving gender, grade and mean interest levels were encouraging. Although it has been previ- 
ously suggested that girls have lower interest in physics than boys (Haussler et al., 1998), it was not noted in this 
study apart from the two individual interest questions. It would also have made sense that 9°" graders would 
have lower interest than 8" graders due to having already learned the topics covered in the teaching module 
(Palmer, 2004), but it is possible that the high novelty of the experiments compensated for the low novelty of 
knowledge. The fact that instructional activities did not predict situational interest was surprising. Experiment- 
ing is usually considered as an interesting activity (Abrahams & Millar, 2008), but in this study, it could not be 
distinguished from other activities, such as listening and answering the worksheet questions, which are arguably 
less interesting than experimenting. It is possible that instructional activities could be distinguished by using 
a larger sample but judging from the findings of Lavonen et al. (2021) and Vilhunen et al. (2021), it depends 
more on the context and topic than sample size (Haussler, 1987). If a ranking of instructional activities based on 
interest is the desired outcome, asking students to explicitly rank them instead of comparing the interest levels 
is likely a better approach. Similarly, to the findings of Nguyen et al. (2018), student interaction with their peers 
alone and not with the teacher was not a significant predictor of situational interest. 

Results about the relation between individual and situational interest are partially in line with previous 
studies on situational interest. Post-problem situational interest levels (measures 2 and 3) were significantly 
higher than individual interest levels (Rotgans & Schmidt, 2017). Similarly to the findings of Rotgans and Schmidt 
(2018), there were positive correlations between subsequent situational interest measures, with measures more 
separated in time having a lower correlation. However, whereas in their study the role of individual interest de- 
creased with time, an opposite trend was noted in this study. The correlation between individual and situational 
interest was low at the beginning of the module and increased with time. The fitted linear regression models 
confirmed the complexity of the relation - the first measure of situational interest was a stronger predictor of 
the second measure of situational interest than individual interest, but the latter was again a stronger predic- 
tor of the third measure of situational interest. Rotgans and Schmidt suggested that “individual interest only 
determines situational interest in the absence of a situationally arousing event” (p. 535). It is hard to believe this 
explains the noted trend, as the experiments the students conducted were certainly novel to them, considering 
that the topic of oscillations is often overlooked by Estonian teachers and in the best cases simple experiments 
involving pendulums are used. As such, the noted trend requires further investigation. 

As the response rate was not significantly correlated with other measures apart from the group, other expla- 
nations are required. It is likely that the students who were excluded from the analysis were not interested in the 
topic of the module or participating in the study. For the included students, the drop in response rates could be 
explained by fatigue and the available option to skip answering. Another possibility is that these students were 
more engaged with experimenting and considered answering not a priority. The non-native group, which had 
the highest share of excluded students and the lowest response rates, could have had language issues. In Estonia, 
these students receive lessons both in Estonian and their mother tongue, but as some concepts in physics are 
difficult to understand even for native speakers, this difficulty was likely amplified. Another question to consider 
is how to handle the incomplete answers. In this study, such data was not used, but another option would be 
to assume that such students had very low levels of interest. However, it is not certain that the lack of interest 
was the sole reason behind not answering. As mentioned in the methodology section, each student received 
their questions at a unique time and had the option to skip answering. While such a design did little to impede 
the workflow, it put much more responsibility on the students themselves to notice the questionnaires and 
deal with them. Dejonckheere and Erbas, (2021) have pointed out that the unpredictability of the questionnaire 
appearance can result in a higher burden and lower compliance. A design where all the students answer at the 
same time would have likely provided more responses, on the account of interrupting the workflow. However, 
it is still important to keep students from guessing when questions are asked, as it can affect their responses. 
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Conclusions and Implications 


The aim of this study was to predict situational interest using gender, grade, individual interest and par- 
ticularly the instructional activities the students engaged in. A related goal was to assess the suitability of the 
chosen data collection method based on the students’ response rate. Individual interest was a strong predictor 
of situational interest, but contrary to previous studies its correlation with situational interest increased with 
time, not decreased. The cause of this trend is unknown and requires further studying. Previous measures of 
situational interest were also significant in predicting the next measures. There were no differences between 
genders and grades in individual or situational interest. Instructional activities were not a significant predictor 
and could not be distinguished from one another based on situational interest. This is likely not connected to 
the sample size but to the context and topic in which the activities were situated. 

As the module progressed, a decline in the situational interest questions’ response rates was noted. Possible 
explanations for this decline are fatigue, lack of interest, a language barrier, and the way the data collection was 
organised. While the chosen data collection method had little impact on the workflow, it negatively affected the 
amount of collected data. In future, it is recommended to avoid having each student answer their questions at 
unique times, as answering simultaneously can help alleviate the responsibility and burden put on students. 
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Abstract. The performance of agricultural 
science students is consistently low as 

seen in their external examinations. This 
could be attributed to the poor teaching 
methods, materials and techniques 
applied in teaching the subject. This study 
determined the multimedia application 
effects on the academic achievement of 
Agricultural science students. It adopted 

a mixed-method design. All the students 
and teachers of Agricultural science in 
secondary schools in Enugu State, Nigeria 
formed the population but six schools with 
583 students and teachers of agriculture 
formed the sample. Students’ Academic 
Achievement Test in Agricultural Science 
(SAATAS) and Convenience Teaching 
Method Questionnaire (CTMQ) were 

used to collect data. Mean, ANCOVA and 
Friedman's test were used for data analyses. 
Results indicated that students instructed 
using multimedia significantly performed 
higher than the students instructed using 
the conventional teaching method. The 
performance of males was not different 
from that of the females. The interaction 
between the teaching media and gender 
had a non-significant effect on the academic 
achievement of students. Moreover, 
teachers adopted note-taking, lecture, 
demonstration and project methods but did 
not use multimedia in teaching. Teachers 
of agriculture accepted that inconsistent 
power supply, poor network connectivity, 
and non-availability of computers, among 
others, were the problems hindering them 
from using multimedia in teaching. 
Keywords: academic achievement, mixed 
method design, multimedia, agricultural 
science, teaching and learning 
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Introduction 


The nature of secondary education at present in Enugu State, Nigeria 
lacks the needed quality for the survival of the recipients in the modern 
economy of the developing world. This is more worrisome as the educational 
policy is more interested in quantity than quality (Thom-otuya & Inko-tariah, 
2016). Ninety-four per cent of secondary school graduates lack the entrepre- 
neurial skills needed to become independent members of society (Asogwa 
et al., 2021). This was further buttressed in the WAEC Chief Examiner's report 
where only 48%, 42.5% and 46% of secondary school students passed in 
2017, 2018 and 2019 senior school certificate examinations respectively. 
The poor secondary school education quality hampers the development of 
effective manpower necessary for national development in some developing 
countries (Ajagbawa, 2014). Secondary education in agricultural science is 
aimed at providing technical knowledge and skills necessary for agriculture, 
commercial and economic development (Federal Republic of Nigeria [FRN], 
2013). Poor quality secondary education can be attributed to inadequate 
teacher qualifications and wrong teaching methods (Arong & Ogbadu, 2010). 
Qualified teachers should have both content and pedagogical knowledge 
to impart worthwhile skills to students (Okwduba & Okigbo, 2018). Content 
knowledge refers to the mastery of the subject matter while pedagogical 
knowledge refers to the mastery of the teaching methods needed in teach- 
ing the content. Competency in the subject matter as well as the adoption 
of suitable instructional methods are essential to learners’ academic perfor- 
mance. Learners’ academic performance is an indication of quality teaching 
which is dependent on the methods, materials and techniques of teaching 
applied (Ganyaupfu, 2013). 
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Unfortunately, some teachers apply instructional methods convenient to them without taking cognizance 
of the content, learners’ needs and interests. Osinem (2008) noted that teachers apply such methods as discus- 
sion, lecture, demonstration, and project, among others as convenient to them without evaluating if the method 
is appropriate to the students or not. A teaching method is appropriate when it employs suitable training values 
which stimulate learners’ activities (Pooja, 2017). The adoption of conventional teaching methods led to the poor 
academic performance of learners in some subjects (Muema et al., 2018). A classroom situation where the teacher 
dominates the class by teaching the students the subject matter without the active involvement of the students 
does not motivate learning. In some schools, students just copy notes from the board without active engagement 
with the subject being taught (Khurshid & Ansari, 2012). It was noted that the poor performance of the Ghanaian 
Senior High School students in Biology was the inability of the teachers to adopt the procedures recommended 
in the syllabus, inability to organize practical work in groups and inability to embark on fieldtrip where students 
can observe the organisms in their natural environment (Safo-Adu et al. 2020). 

Therefore, student-centred teaching methods should be emphasized in schools if educational quality 
must be promoted to meet the technological requirements for developing countries’ national development. 
One of such methods includes the application of multimedia presentation. The application of multimedia during 
instructional delivery allows the student to interact with the content. However, Molla and Muche (2018) reported 
that inadequate knowledge and skills in the application of computer software by teachers result in their inability 
to utilize multimedia in teaching. The absence of multimedia training is the greatest problem for its’ utilization for 
instructional delivery. Sarowardy and Halder (2019) reported that the negligence of teachers in the application 
of multimedia was a result of insufficient digital amenities and poor infrastructural facilities in schools. On several 
occasions, teachers were requested to undergo training without undergoing a needs analysis to find out those 
who needed the training and the sort of training needed. 

When teachers are upskilled in the application of multimedia in instructional delivery, it will be easier 
to deliver a learner-friendly lesson that would guarantee comprehension. When students are actively engaged, 
they will certainly learn, and the knowledge will be ploughed back for economic development. In today’s global 
economy that is knowledge-driven, the use of multimedia in teaching especially in developing countries builds 
students’ intellectual capacity and knowledge that will help them compete favourably with their peers in every 
part of the world as they contribute to national and international economic development. 

This study, therefore, sought to determine the effects of multimedia application on the academic achieve- 
ment of students in Agricultural Science, the influence of gender on the academic achievement of students in 
Agricultural Science, the interaction effect of different teaching media and gender on students’ academic achieve- 
ment in Agricultural Science, convenient teaching methods utilized by the teachers and problems hindering the 
application of multimedia in teaching. 


Research Questions 


1. Whatare the effects of multimedia application on the academic achievement of students in agricultural 
science? 

2. What is the influence of gender on the academic achievement of students in agricultural science? 

3. Whatare the interaction effects of different teaching media and gender on the academic achievement 
of students in agricultural science? 

4. What are the convenient teaching methods utilized by the teachers of agriculture in teaching agricul- 
tural science? 


Research Hypotheses 
1. There is a significant effect of multimedia application on the academic achievement of students in 
agricultural science. 
2. Thereisa significant influence of gender on the academic achievement of students in agricultural science. 
3. Thereisa significant interaction effect of different teaching media and gender on the student's academic 
achievement in agricultural science. 
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Literature Review 
Multimedia Use in Teaching 


Media in education involves all the avenues to produce, save and deliver instruction using an instrument like 
a chalkboard, texts, computer, projectors, cameras, audio facilities, sound and video systems combined (de Sousa 
et al., 2017). Multimedia is, therefore, the application of many channels in communicating particular information. 
Multimedia according to Gunawardhana and Palaniappan (2016), is the numerous media elements such as texts, 
graphics, animations, video, sound, etc. integrated as one whole, which are helping users. When all these media 
are combined, it makes communication clearer and easier. Almara’beh et al. (2015) affirmed that multimedia can 
be an integration of several mass media like text, audio and video and can also be the production of computer pro- 
grammes in a large quantity that individuals can use for studying. Learning can be result-oriented, more inclusive, 
without time, distance and space barriers, tailor-made to personal style, and increasingly collaborative between 
students and their teachers when multimedia is involved. 

Multimedia is very useful due to its interactivity, pliability, and the use of several media to advance learning, 
taking into account the different learning styles of the students which motivate their interest (Andresen & Brink, 
2013). Multimedia applications in the view of Babiker (2015) are classified into text-based, interactive, web and 
mobile (smart) phone applications. Babiker emphasized that most educational media are categorized as interac- 
tive and graphical applications as it is a good tool that is used in all media formats. When compared with other 
means of instructional delivery, Patel (2013) noted that multimedia shall be a successor to conventional teaching 
methods. The author emphasised that multimedia has several options for making teaching interactive and more 
result-oriented. Conventional teaching methods are not famous while multimedia with fantastic audio and visual 
animation conveniently allows greater access to information and presents a natural environment which motivates 
students’ interest (Patel, 2013). Mukherjee (2018) noted that multimedia improves students’ ideation and practical 
language skills that are essential for quality education. The application of multimedia could be further developed 
by investing in the facilities and educating the teachers on the use of the facilities (Gorgoretti, 2019). 


Teaching Methods 


Conventional teaching methods like the lecture method are not activity-based and, therefore, do not stimu- 
late learners to acquire solutions to real-life problems based on practical knowledge (Ganyaupfu, 2013). Teachers, 
according to Ganyaupfu, may try to increase the passage of the content while reducing time and effort so that 
students may learn. In the conventional lecture method, students are not properly engaged and concentration 
fades after some time. Kola and Langenhoven (2015) noted that the demonstration method is good for practical 
skills but there is limited time to perform a demonstration in the classroom. In the same vein, the study by Molla 
and Muche (2018) reported that cooperative learning is an appropriate teaching method but is affected by the 
classroom size, particularly the number of students and the shortage of instructional materials. Implementation 
of inquiry-based learning is strenuous largely due to the insufficient laboratory equipment, instructional aids, 
time available and a large number of students in a class (Ramnarain & Hlatswayo, 2018). An appropriate teaching 
method assists the learners in asking questions about what they knew earlier which will help them to learn, by 
compelling them to find answers to their questions, as well as the instrument of responsibilities for change (Shirani 
et al., 2016). With the audio, video, picture, and animation features of multimedia, learning difficult and complex 
topics are easy to understand. 


Students’ Academic Achievement 


The academic achievement of students measures the learning outcomes of students who received a particular 
instruction. The achievement of students is dependent on many factors such as teaching methods, instructional 
materials, teachers’ competence, and teaching environment, among others. Podunggee et al. (2019) noted that 
teachers’ competence positively influences students’ academic achievement by increasing the students’ intrinsic 
motivation to study. Nbina (2012) reported a strong correlation between teachers’ competence and students’ 
academic achievement in Chemistry. Teachers’ subject content knowledge and pedagogical skills were reported to 
have a significant influence on Junior Secondary school students’ academic achievement (Amie-Ogan & Etuk, 2020). 


1076 


Ass https://doi.org/10.33225/jbse/23.22.1074 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


= i EFFECTS OF MULTIMEDIA APPLICATION ON STUDENTS’ ACADEMIC AGHIEVEMENT IN 
ISSN 1648-3898 | /print/ Pees SA 
ISSN 2538-71 38 /Online/ (PpP.1074-1088) 


When teachers apply appropriate methods and materials in teaching, students are motivated to learn. Aydisheh 
and Gharibi (2015) noted that the constructivist teaching method influenced the cognitive ability of students 
when appropriately applied. In the analysis of the effectiveness of different instructional methods on the academic 
achievement of learners, Ganyaupfu (2013) observed that the teacher-student interactive instructional method 
was the most efficient, which was followed by the student-centred teaching method while the teacher-centred 
teaching method was the least efficient in enhancing students’ academic achievement. Bayrak and Bayram (2010) 
noted that the computer-aided instructional method positively affected the academic achievement of students in 
Science and Technology when teaching acid-base subjects. Oladosu et al. (2022) observed a significant relationship 
between the professional development of teachers, the quality of teachers and students’ academic achievement 
in publicly and privately owned schools. 


Theoretical Framework 


This study adopted the cognitive theory of multimedia learning by Richard Mayer in 2005. The theory stipulates 
that learning is active and effective when words and pictures are integrated into the lesson. Active learning can be 
attained through the combination of sounds and images for a wider experience (Ejimonye et al., 2020). Students 
learn using different styles such as visual, aural, reading and writing but learning can be optimized when a combina- 
tion of these styles is presented (Gilakjani, 2012). Presentation of a lesson using the multimedia teaching method 
lends learners the opportunity to observe the images, texts and movies and also hear the sound or voice, which 
offers a meaningful experience to the learners. Designing a multimedia environment that appeals to visualization 
by making the background look attractive, adding pictures, icons, graphics, and bold colour texts combined with 
voice improves learning in students (Sukma et al., 2018). When students are taught using multimedia, they will 
apply their eye for seeing, ear for hearing, mind for feeling and hence the brain for proper coordination of all the 
activities. The study is in alignment with Richard Mayer’s theory because it involves the electronic presentation 
of lessons where graphics, sounds and texts are integrated to appeal to the various sense organs of the students. 


Research Methodology 
Design 


Amixed-method design was adopted in this study. The mixed-method design combines two or more different 
designs in carrying out a study (Doye et al., 2009). The design was appropriate for the study since quasi-experimental 
and survey designs were adopted. 


Ethics 


The approval to conduct the study was granted by the University of Nigeria Research Ethics Committee (UNN- 
REEC-22-059) on January 22, 2022 (UNN-REEC-22-059). Before the commencement of the study, a written request 
was sent to the administrators of the participating schools and permission was obtained to conduct the study in 
the respective schools. Furthermore, the consent of the Agricultural science teachers was obtained through the 
letters delivered by the researchers which were read and the request to be part of the study was granted. 


Participants 


The population of the participants was 11,145 representing 10,385 agricultural science students from 55 
secondary schools and 760 teachers of agriculture in Enugu State, Nigeria (Enugu State Ministry of Education, 
2022). The sample size was 583 representing all the agricultural science teachers and students in the randomly 
selected schools. A random sampling technique was adopted to select six schools (two schools each from the three 
educational zones in the state) where all the 560 students and 23 teachers of agriculture present in the schools 
were studied. The six randomly selected schools have the students’ distribution of 93, 82, 105, 96, 84 and 100, and 
teachers’ distribution of 4, 3, 5, 3, 3 and 5 respectively. The entire 583 agricultural science teachers and students 
were used to avoid sampling error and to have a reasonable size that would generate a reasonable result. 
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Instruments and Procedure 


Two research instruments were developed to collect data. Students’ Academic Achievement Test in Agricul- 
tural Science (SAATAS) was developed to assess the performance of students while Convenience Teaching Method 
Questionnaire (CTMQ) was developed to ascertain the most convenient teaching method applied by the teachers. 
At the end of the questionnaire, an open-ended question was asked to find out the problems teachers of agricul- 
tural science faced in using the multimedia presentation. 

A topic was chosen from the curriculum of senior secondary two (SS2) agricultural science. The topic was 
Animal Nutrition. Animal nutrition was chosen because it was the topic due for study by the SS2 students when the 
research commenced. Similarly, SS2 was chosen because it was the most stable class since $S3 students were in their 
certificate examination preparation and might not be ready to participate in the study. Sixteen lesson plans were 
developed to teach the topic in eight weeks using multimedia presentation (experimental group) and conventional 
teaching methods (control group) respectively. The lesson plans were developed based on the preliminary informa- 
tion (date, school, class, age of students, duration, topic), set induction, entry behaviour, instructional materials, 
specific objectives, instructional procedure, evaluation, summary, and conclusion (Osinem, 2008). The lesson plans 
were face validated by five experts from the Department of Agricultural Education, University of Nigeria, Nsukka. 

The developed SAATAS was guided by the test-blue print developed by the researchers. The test-blue print 
was developed following the levels of cognitive educational objectives proposed by Benjamin S. Bloom (Nworgu, 
2003), which include knowledge, comprehension, application, analysis, synthesis, and evaluation (Table 1). The 
development of tests using a test-blue print is a means of ensuring content validity as it helps in constructing 
unbiased and valid tests (Cohen et al., 2007). 


Table 1 
Test-Blue Print of the Multimedia Application on Students’ Academic Achievement 


Knowledge § Comprehension Application Analysis Synthesis Evaluation 


Contents 30% 25% 20% 10% 10% 5 io 
Digestion in Farm 4 4 3 1 1 1 14 
Animals 20% 

Classification of Livestock 3 3 2 1 1 1 11 
Feed 15% 

Food Nutrients 15% 3 3 2 1 1 1 11 
Types of Ration and 4 4 3 1 1 1 14 
Uses 20% 

Ration Formulation 20% 4 4 3 1 1 1 14 
Malnutrition in Farm 2 2 1 1 1 0 7 
Animals 10% 

Total 20 20 14 6 6 5 71 


The developed SAATAS contained 71 question items where the objective levels of knowledge, comprehen- 
sion, application, analysis, synthesis and evaluation contained 20, 20, 14, 6, 6 and 5 questions respectively while 
the subtopics made up of digestion in farm animals, classification of livestock feed, feed nutrients, types of ration 
and uses, ration formulation and malnutrition in farm animals contained 14, 11, 11, 14, 14 and 7 questions respec- 
tively (Table 1). The SAATAS was face validated by five experts, selected from the Departments of Measurement 
and Evaluation, Animal Science and Agricultural Education, all from the University of Nigeria, Nsukka. 

Furthermore, the test was pilot tested on 40 students of Agricultural Science from Ebonyi State, Nigeria on 
two separate occasions to determine the reliability using the test-retest method. The result was subjected to the 
Pearson Product Moment correlation coefficient which had a value of .73. The internal consistency of the instrument 
was also computed using Kuder-Richardson 20 and it had a value of .78. Moreover, the psychometric properties of 
the test were computed from the same pilot-test which yielded a discriminatory index of d=.45, difficulty index of 
p =.68 and a distractor index of D = .09. The results showed that the test items were appropriate as all the indices 
were positive and the magnitudes were high. 
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The CTMQ was pilot tested on 40 teachers of Agricultural Science in Anambra State, Nigeria. The reliability was 
then computed using Cronbach's alpha method which had a value of a = .82. The questionnaire was structured on 
a 5-point Likert scale ranging from strongly agree to strongly disagree with the value range of 5 to 1. 

In the procedure of the experiment, the six randomly selected schools were randomly allotted to two treat- 
ment groups of three replicates each. Three schools representing the control group were instructed in animal 
nutrition using the conventional teaching method whereas the other three schools representing the experimental 
group were instructed in animal nutrition using the multimedia presentation. The classes were not manipulated 
but were used as intact classes. The SAATAS was administered as the pre-test before the beginning of the instruc- 
tion. The same instrument was also administered as the post-test after the instruction but, the item numbers and 
the options were reshuffled to make the instrument appear different. The instruction lasted for two months. The 
CTMQ was administered to the 23 agricultural science teachers in the sampled schools by the researchers and 
retrieved a week later. 


Data Analysis 


Data collected from teaching students with different teaching media were analysed by mean and analysis of 
covariance (ANCOVA) which was tested at a .05 level of significance. Friedman test (r) was applied to analyse the 
data collected with the CTMQ by ranking the teachers’ opinions on the teaching methods adopted in teaching 
agricultural science. The result of the mean was interpreted following the real limit of numbers as in the table below. 


Table 2 
Real Limit of Numbers for Taking Decision on the Mean 


Real Limit Value Decision 
5 to 1.49 1 Strongly Disagree 
1.50 to 2.49 2 Disagree 
2.50 to 3.49 3 Undecided 
3.50 to 4.49 4 Agree 
4.50 & above 5 Strongly Agree 


Any item with a mean of 4.50 and above was interpreted as strongly agree. Similarly, any item with the mean 
range of 3.50 to 4.49 was interpreted as agree, any item with the mean range of 2.50 to 3.49 was interpreted as 
undecided, any item with the mean range of 1.50 to 2.49 was interpreted as disagree and any item with the mean 
range of .50 to 1.49 was interpreted as strongly disagree. The data collected from the problems faced by the teach- 
ers of agriculture in using multimedia applications were categorized based on related problems. The frequencies 
of the categories were counted, and data were subjected to percentages. 


Research Results 
Effects of Multimedia Application on the Students’ Academic Achievement 


Results on the effect of the multimedia application on the Students’ academic achievement are presented in 
Tables 3 and 4. 
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Mean Academic Achievement of Agricultural Science Students Taught Using Different Teaching Media 


Pre-test Post-test 
Teaching Media oo X Gain 
X SD X SD 
Multimedia (experimental group) 38.83 10.45 84.74 14.07 45.91 
Conventional (control group) 35.11 10.56 55.23 15.49 20.12 
X Gain 3.72 29.51 


The students who received instruction through the application of multimedia had a pre-test mean score of 
38.83+10.45 while the students taught agricultural science using the conventional method had a mean pre-test 
score of 35.11+10.56 with a negligible mean gain of 3.72 (Table 3). It was found that students who received agri- 
cultural science instruction through multimedia application had a mean post-test score of 84.74+14.07 while the 
students taught agricultural science using the conventional method had a mean post-test score of 55.23+15.49 
with a mean gain of 29.51 in favour of the multimedia application. Similarly, the students taught agricultural science 
using multimedia applications had a mean pre-test/post-test difference of 45.91 as against the students taught 
agricultural science using the conventional method with a mean pre-test/post-test difference of 20.12 (Table 3). 
The result shows that the students of agricultural science had a mean gain achievement of 45.91 after receiving 
instruction through multimedia whereas the same group had a mean gain achievement of 29.51 over those that 
received agricultural science instruction through the conventional method. This implies that multimedia applica- 
tions had a better academic achievement in the students of agriculture than the conventional teaching method. 


Table 4 
ANCOVA of the Effects of Multimedia Application on the Academic Performance of Students in Agricultural Science 


Source aie a df Mean Square F p-value yah 
Corrected Model 12384.609a 2 6192.305 27.221 <.001 489 
Intercept 22575.663 1 22575.663 99.243 <.001 635 
Pre-test 22.381 1 22.381 .098 155 002 
Teaching Media 12174.726 1 12174.726 53.520 001 484 
Error 12966.324 143 227.479 
Total 290952.000 146 
Corrected Total 25350.933 145 


Note. *R Squared = .489 (Adjusted R Squared = .471) 


A statistically significant difference existed (p = .001<.05, F = 53.52) in the mean achievement of the students 
who received instruction through multimedia applications and conventional teaching methods (Table 4). The result 
equally shows that a moderate effect size of 48.4% existed and could be attributed to the multimedia application. 
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Influence of Gender on the Academic Achievement of Students 


Results on the influence of gender on the academic achievement of agricultural science students are pre- 
sented in Tables 5 and 6. 


Table 5 
Mean Academic Achievement of Male and Female Students in Agricultural Science 


Pre-test Post-test 
Gender 7 ae X Gain 
xX SD xX SD 
Male 36.35 11.69 66.35 21.06 30 
Female 36.68 9.83 66.68 20.79 30 


The result presented in Table 5 shows that the male students who received instruction through multimedia 
and conventional teaching methods had a mean pre-test score of 36.35+11.69 and a post-test mean score of 
66.35+21.06 with a mean achievement gain of 30. Similarly, the female students who received instruction through 
multimedia and conventional teaching methods had a pre-test mean score of 36.68+9.83 and a post-test mean 
score of 66.68+20.79 with a mean achievement gain of 30. This implies that males did not perform better than 
females when multimedia and conventional methods were used. 


Table 6 
ANCOVA of the Effect of Gender on the Academic Achievement of Students in Agricultural Science 


Source enn a df Mean Square F p-value Partial Eta Squared 
Corrected Model 997.306a 2 498.653 1.167 319 .039 
Intercept 12528.169 1 12528.169 29.322 <.001 340 
Pre-test 995.699 1 995.699 2.330 .132 .039 
Gender 602 1 602 001 .970 <.001 
Error 24353.627 143 427.257 
Total 290952.000 146 
Corrected Total 25350.933 145 


Note. @R Squared = .039 (Adjusted R Squared = .006) 
The result in Table 6 shows that gender did not have a statistically significant difference (p= .97 > .05, F=0.001) 


in the student's academic achievement in Agricultural Science. No significant effect size (0%) was equally found 
to exist. 


Interaction Effect of Different Teaching Media and Gender on Students’ Academic Achievement 


Results on the interaction effect of different teaching media and gender on the academic achievement of 
students are presented in Tables 7 and 8. 
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Mean Interaction Effects of Different Teaching Media and Gender on the Academic Achievement of Students in Agricultural 
Science 
Pre-test Post-test _ 
Teaching Methods Gender ar X Gain = Salhi 
= SD = SD Difference 
Male 42.86 11.70 92.14 6.77 49.28 
Multimedia 4.84 
Female 37.07 9.72 81.50 15.34 44.43 
Male 33.95 11.03 56.84 15.70 22.89 
Conventional 5.72 
Female 36.33 10.20 53.50 15.52 17.17 


The male students who received agricultural science instruction through multimedia application had a mean 
pre-test achievement score of 42.86+11.70 and a mean post-test achievement score of 92.14+6.77 with a mean 
gain score of 49.28 (Table 7). The female students who received agricultural science instruction through multime- 
dia application had a mean pre-test achievement score of 37.07+9.72 and a mean post-test achievement score of 
81.50+15.34 with a mean gain score of 44.43. In the same vein, the male students who received agricultural science 
instruction through the conventional teaching method had a mean pre-test achievement score of 33.95+11.03 and 
the mean post-test achievement score of 56.84+15.70 with a mean achievement gain of 22.89 while the female 
counterpart had the mean pre-test achievement score of 36.33+10.20 and the mean post-test achievement score 
of 53.50+15.52 with a mean gain score of 17.17 (Table 7). It was observed that the mean difference between the 
male and female students that received agricultural science instruction through the multimedia application was 
4.84 while the mean difference between the male and female students that received agricultural science instruc- 
tion through the conventional teaching method was 5.72. The mean difference between the male and female 
agricultural science students who received instruction through multimedia applications and conventional teach- 
ing methods was 0.88. 


cous of the Interaction Effect of Different Teaching Media and Gender on the Academic Achievement of Students in Ag- 
ricultural Science 
Source Liao a df Mean Square F p-value ‘saned 
Corrected Model 13032.105a 4 3258.026 14.546 <.001 514 
Intercept 19351.823 1 19351.823 86.400 <.001 611 
Pre-test 15.055 1 15.055 .067 196 .001 
Teaching Media 12014.245 1 12014.245 53.640 .001 494 
Gender 605.255 1 605.255 2.702 106 047 
Teaching Media * Gender 146.688 1 146.688 .655 422 012 
Error 12318.829 141 223.979 
Total 290952.000 146 
Corrected Total 25350.933 145 


Note. @R Squared = .514 (Adjusted R Squared = .479) 
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Non-significant interaction effect (p=.42>.05, F=.66) of different teaching media and gender on students’ 
academic performance in Agricultural Science was found (Table 8). The table showed that there was a significant 
difference (p=.001<.05, F=53.64) in students’academic achievement in Agricultural Science when teaching media 
was considered individually and a statistically significant difference did not exist (p=.11>.05, F=2.70) in students’ 
academic achievement in Agricultural Science when gender was considered separately. The effect size of the 
interaction between the different teaching media and gender was negligible (1.2%) (Table 8). This implies that 
teaching media and gender do not collectively have any effect on students’ academic achievement. 


Convenient Teaching Methods Utilized by the Teachers and Problems Hindering the 
Application of Multimedia in Teaching 


Results on the convenient teaching methods utilized by the teachers and problems hindering the application 
of multimedia in teaching are presented in Table 9. 


Table 9 
Convenient Teaching Methods Adopted by the Teachers of Agricultural Science in Teaching the Subject 


Teaching Methods xX r value 
Discussion 3.23 3.56 
Lecture 3.88 4.12 
Demonstration 3.90 4.11 
Multimedia Presentation 2.11 2.35 
Project 3.59 4.03 
Problem Solving 2.99 3.32 
Note Taking 3.71 4.49 


Teachers of Agricultural Science ranked the note-taking method as the most appropriate method conve- 
nient in teaching agricultural science (r = 4.49), followed by the lecture method (r = 4.12) and demonstration 
method (r= 4.11) (Table 9). Similarly, teachers of agricultural science ranked the multimedia presentation as 
the least method (r = 2.35) used in instructing agriculture in schools followed by the problem-solving method 
(r = 3.32). The result presented in Table 9 also revealed that teachers of agricultural science agree that they use 
demonstration, lecture, note-taking, and project methods (X = 3.90, 3.88, 3.71, 3.59) in teaching agricultural 
science but disagree that they use multimedia presentation ( = 2.11) in teaching agricultural science (Table 9). 
The result shows that the teachers were undecided whether they use discussion and problem-solving methods 
or not (X = 3.23, 2.99) in teaching the subject. 

Results of the problems faced by the teachers of agriculture on the use of multimedia in teaching revealed 
that the teachers were faced with the challenges of inconsistent power supply (15, 65.23%), poor network con- 
nectivity (17, 73.91%), non-availability of computer sets in most schools (16, 69.57%), poor computer usage by 
the teachers (20, 86.96%) and poor staff development in schools (21, 91.30%). 


Discussion 


It was found that the multimedia application increased the students’ academic achievement greater than 
the conventional teaching method. The findings equally showed that a statistically significant difference existed 
(p < .05) in the mean academic achievement of the learners who received instruction through the applica- 
tion of multimedia and conventional teaching method in favour of the multimedia application. The greater 
performance of students taught using multimedia might be attributed to the varied features of multimedia 
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where students are exposed to the lesson content using the combination of pictures, audio and video which 
appeals to the eye, ear, head, and heart. The result was in agreement with Ezennia et al. (2016) who found out 
that using visual communication tools of multimedia is among the most efficient means of clarifying complex 
concepts to students for easy understanding. In the teaching and learning of communication skills, Mukherjee 
(2018) found out that multimedia improves students’ ideation and practical language skills that are essential 
for quality education. In a study on the effect of multimedia on students’ academic performance in geography, 
Incedayi (2018) reported that using multimedia in teaching was better than the conventional method of im- 
parting knowledge to the learners. Multimedia presents lessons using various modes to prepare a competent 
workforce capable of contributing to the economic productivity of the nation (Kozma, 2008). When compared 
with other means of instructional delivery, Patel (2013) noted that multimedia shall be a successor to conven- 
tional teaching methods. Well-prepared students impact positively on their surroundings. They are agents of 
change and future influencers. They comprehend peoples’ intentions and actions and contribute maximally to 
economic development (Organisation for Economic Co-operation and Development [OECD], 2018). 

It was equally found that males did not do better than females in agricultural science when taught 
through the multimedia application. Similarly, statistically significant differences did not exist (p > .05) in the 
mean academic achievement of male and female students who received instruction through the multimedia 
application. This might be as a result that male and female students have no significant cognitive ability differ- 
ent from the other. The result had the support of Umar et al. (2015) who reported a non-significant difference 
in the academic achievement of males and females in Colleges of Education in Borno State. It was found in 
another study that the academic potentials of male and female pharmacology students were the same (Faisal 
et al., 2017). Though there was a slightly better achievement of female medical students’ education than the 
male counterparts concerning GPA, this, however, cannot be confirmed that male students perform better than 
females in other undergraduate programmes (Albalawi, 2019). The result contrasts with the findings of Saida 
and Mustapha (2018) who reported that female students got better grades in most undergraduate courses in 
agriculture when compared with males. 

The findings on the interaction effect of gender and different teaching media showed a slight mean differ- 
ence in the achievement of male and female students who received instruction through the different teaching 
media. However, no statistically significant interaction effects existed (p > .05) between gender and teaching 
media on students’ academic achievement. This result might be attributed to the same cognitive ability of 
male and female students where both perform equally when they are exposed to the same content using the 
same materials and methods in the same environment. In agreement with the current findings, Okeke (2018) 
found no statistically significant interaction effect of gender and Mend Mapping Teaching Strategy (MMTS) 
on students’ academic performance in chemistry. The author reported a significant difference in the academic 
performance of students taught using MMTS and conventional methods, but the interaction effect was not 
found. A similar study reported no significant interaction effects of teaching methods, gender, and study habit 
on the learners’ academic performance in Basic Science (Gbenga & Effiong, 2015). This signifies that teaching 
methods, gender and study habit did not influence the student’s academic performance in Basic Science when 
combined. Similarly, Ajayi and Ogbeba (2017) discovered that no significant interaction was found between 
gender and hands-on activities on the academic achievement of learners in Stoichiometry. In another study, 
Zudonu and Njoku (2018) reported no interaction between gender and teaching methods on students’ at- 
titudes to chemistry. 

The findings further revealed that teachers of agricultural science favour the use of some teaching methods 
like note-taking, lecture, demonstration and project methods at the expense of the application of multimedia 
presentation in teaching. The findings might be attributed to the non-compliance of agricultural science teach- 
ers to information and communication technology (ICT) applications. Innovative teaching method which most 
teachers fail to use in this computer-mediated education era helps to generate interest, motivate students’ 
greater involvement, and raise their curiosity (Mahajan & Kaushal, 2017). Pooja (2017) reported that teachers 
predominantly use demonstration, recitation, and memorization in teaching. The appropriateness of a chosen 
method is highly instrumental to the performance of students and hence, should consider the individual dif- 
ferences of the learners which the application of ICT can take care of. An appropriate teaching method assists 
the learners to ask questions about what they knew earlier which will help them to learn, by compelling them 
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to find answers to their questions (Shirani et al., 2016). Ganyaupfu (2013) noted that many teachers applied 
teacher-centred methods like note-taking and lecture methods in imparting worthwhile knowledge to the 
students. Teacher-centred approach is deficient in allowing students’ participation and active engagement in 
the learning process. 

Furthermore, the findings on the open-ended question revealed that the teachers were faced with the 
challenges of inconsistent power supply, poor network connectivity, non-availability of computer sets in most 
schools, poor computer usage by the teachers and poor staff development in schools. This might be the result 
of poor funding for education which bedevils developing countries’ educational systems. The findings were 
not different from the work of Sarowardy and Halder (2019) which reported that the negligence of teachers 
in the application of multimedia was a result of insufficient digital amenities and poor infrastructural facilities 
in schools. Some areas where schools are located do not have an electricity supply to power computers. Neo 
and Neo (2001) added that an adequate number of computers must be made available and proper training of 
teachers on the use of the multimedia application should be done before they embrace the technology. Teach- 
ers lack computing skills and therefore find it difficult to apply multimedia in teaching and learning (Zhang, 
2016). Bayrak and Bayram (2010) noted that the computer-aided instructional method positively affected the 
academic achievement of students in Science and Technology, but the application was restricted by insuffi- 
cient numbers of computer facilities in the school. The application of multimedia in instructional delivery no 
doubt motivates learners and accelerates their comprehension, but the issue of computer and network access, 
training of teachers in computer skills and their motivation are paramount to the application of multimedia 
in classroom teaching and learning. 


Conclusions and Implications 


The research results confirmed that the application of multimedia in classroom instruction improves 
the academic achievement of students. This proves its potential to arouse curiosity and activate the spirit of 
imagination in students through varied means of presentation such as image, audio, video, animation, sound, 
and colour. The application of multimedia in teaching agricultural science has the same effect on the academic 
achievement of male and female students. This signifies that the cognitive ability of male and female students 
does not differ significantly, and when exposed to the same content at the same time will not perform signifi- 
cantly different. 

Although the study contributed to the knowledge of teaching and learning, some limitations need to 
be taken into consideration. Further study may focus on how to equip the curriculum planners in developing 
countries to integrate the multimedia application into the curriculum and also teachers to apply the media 
in teaching and learning. Secondly, further study needs to be done on how to increase funding of schools in 
developing countries to procure ICT facilities that will encourage the application of multimedia in teaching 
and learning. 


Declaration of Interest 
The authors declare no competing interest. 


References 


Ajagbawa, H. (2014). Quality education and financing challenges in Nigeria. /OSR Journal of Research & Method in Education, 
4(5), 54-75. 

Ajayi, O.V., & Ogbeba, J. (2017). Effect of gender on senior secondary chemistry students’ achievement in Stoichiometry using 
hands-on activities. American Journal of Educational Research, 5(8), 839-842. https://doi.org/10.12691/education-5-8-1 

Albalawi, M. (2019). Does gender difference affect the academic achievements of undergraduate students and later interns? 
A single medical College experience, Taibah University, KSA. Allied Journal of Medical Research, 3(1), 20-25. 

Almara’beh, H., Amer, E. F., & Sulieman, A. (2015). The effectiveness of multimedia learning tools in education. International 
Journal of Advanced Research in Computer Science and Software Engineering, 5(12), 761-764. 


1085 


https://doi.org/10.33225/jbse/23.22.1074 LT 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


EFFECTS OF MULTIMEDIA APPLICATION ON STUDENTS’ AGADEMIC ACHIEVEMENT IN ISSN 1648-3898 /Print/ 
AGRICULTURAL SCIENCE 
(PP.1074-1088) ISSN 2538-7 1 38 /Online/ 


Amie-Ogan, O.T., & Etuk, E. F. (2020). Influence of Teachers’ Competence on Students’ Academic Performance in Public Senior 
Secondary Schools in Ikot Ekpene and Essien Udim Local Government Areas of Akwa lbom State. International Journal 
of Innovative Social Sciences & Humanities Research, 8(4), 118-128. 

Andresen, B. B., & Brink, K. (2013). Multimedia in the education curriculum. Moscow, Russia: UNESCO Institute for Information 
Technologies in Education. https://www.eportal.igbajopolytechnic.edu.ng/pdf/multimedia-in-education-nbte- 
curriculum-67.pdf 

Arong, F.E., & Ogbadu, M. A. (2010). Major causes of declining quality of education in Nigeria from administration perspective: 
A case study of Dekina Local Government Area. Canadian Social Science, 6(3), 183-198. 

Asogwa, V. C., Isiwu, E. C., & Ugwuoke, C. U. (2021). Effects of instructional materials on the students’ academic achievement 
in the fishery in senior secondary schools. Global Journal of Educational Research, 20, 153-161 

Aydisheh, F. H., & Gharibi, H. (2015). Effectiveness of Constructivist Teaching Method on Students’ Mathematics Academic 
Achievement. Mediterranean Journal of Social Sciences, 6, 572-579. https://doi.org/10.5901/mjss.2015.v6n6s2p572 

Babiker, M. E. A. (2015). For effective use of multimedia in education, teachers must develop their educational multimedia 
applications. The Turkish Online Journal of Educational Technology, 14(4), 62-68 

Bayrak, B. K., & Bayram, H. (2010). The effect of computer-aided teaching method on the student's academic 
achievement in the science and technology course. Procedia Social and Behavioural Sciences, 9, 235-238. 
https://doi.org/10.1016/j.sbspro.2010.12.142 

Cohen, C., Manion, L., & Morrison, K. (2007). Research Methods in Education. London, 544 Routledge Taylor & Francis Group. 

De Sousa, L., Richter, B., & Nel, C. (2017). The effect of multimedia uses on the teaching and learning of social sciences at tertiary 
level: A case study. Yesterday and Today, 17, 1-22. https://doi.org/10.17159/2223-0386/2017/n17al 

Doye, L., Brady, A. M., & Byrne, G. (2009). An overview of mixed methods research. Journal of Research in Nursing, 14(2), 175-185. 
https://doi.org/10.1177/1744987 108093962 

Ejimonye, J. C., Ugwuanyi, C. S., Okeke, C. |. 0., & Nwoye, M. N. (2020). Two-dimensional animation and students’ achievement in 
Mathematical Economics: Implications for science teaching. International Journal of Engineering Research and Technology, 
13(6), 1220-1230. https://doi.org/10.37624/IJERT/13.6.2020.1220-1230_ 

Enugu State Ministry of Education, (2022). Enugu State Ministry of Education Statistics. 

Ezennia, I. S., Uwajeh, P. C., Agbonome, P.C., & lyendo, T. O. (2016). The benefits of digital multimedia as a teaching and learning 
in Architectural design studios. International Journal of Current Research, 8(9), 39210-39271. 

Faisal, R., Shinwari, L., & Hussain, S. S. (2017). Academic performance of males in comparison with female undergraduate 
medical students in pharmacology examinations. Journal of the Pakistan Medical Association, 67(2), 204-208 

Federal Republic of Nigeria, (2013). National Policy on Education. NERDC Press. 

Ganyaupfu, E. M. (2013). Teaching methods and students’ academic performance. International Journal of Humanities and 
Social Science Invention, 2(9), 29-35. 

Gbenga, A. J., & Effiong, O. E. (2015). Effects of two methods of teaching on students’ achievement in Junior 
secondary schools in Yakurr, Cross River. International Letters of Social and Humanities Sciences, 61, 70-81. 
https://doi.org/10.18052/www.scipress.com/ILSHS.61.70 

Gilakjani, A. P. (2012). A study on the impact of using multimedia to improve the quality of English language teaching. Journal 
of Language Teaching and Research, 3(6), 1208-1215. https://doi.org/10.4304/jltr.3.6.1208-1215 

Gorgoretti, B. (2019). The use of technology in music education in North Cyprus according to student music teachers. South 
African Journal of Education, 39(1), 1-10. https://doi.org/10.15700/saje.v39n1a1436 

Gunawardhana, L. K. P. D., & Palaniappan, S. (2016). Using multimedia as an education tool. Paper presented at the 9% 
international conference on computer games multimedia and allied technologies, Singapore, 28-29 March. 
https://www.researchgate.net/publication/313512325_Using_Multimedia_as_an_Education_Tool 

Incedayi, N. (2018). The impact of using multimedia technologies on students’academic achievement in the BakirkGy final College. 
International Journal of Humanities Social Sciences and Education, 5(1), 40-47. https://doi.org/10.2043 1/2349-0381.0501007 

Khurshid, F., & Ansari, U. (2012). Effects of innovative teaching strategies on students’ performance. Global Journal of Humanities 
Social Science Linguistics & Education, 12(10), 47-53 

Kola, A. J., & Langenhoven, K. (2015). Teaching method in science education: The need for a paradigm shift to peer instruction 
(PI) in Nigerian schools. International Journal of Academic Research and Reflection, 3(6), 6-15 

Kozma, R. B. (2008). ICT, Education reform and economic growth: A conceptual framework. Intel Corporation. 
https://www.intel.com/content/dam/www/public/us/en/documents/brochures/kozma-wp 1-conceptual-framework.pdf 

Mahajan, A., & Kaushal, K. (2017). Impact of innovative pedagogical teaching methods on students’ academic performance. 
New Nigerian Journal of Clinical Research, 6(10), 41-44. https://doi.org/10.4103/2250-9658.225332 

Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R.E. Mayer (Ed.), The Cambridge handbook of multimedia 
learning. Cambridge University Press. 

Molla, E., & Muche, M. (2018). Impact of cooperative learning approaches on students’ academic achievement and laboratory 
proficiency in Biology subject in selected rural schools, Ethiopia. Educational Research International, 2018, 1-9. 
https://doi.org/10.1155/2018/6202484 


1086 


ASS https://doi.org/10.33225/jbse/23.22.1074 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


= i EFFECTS OF MULTIMEDIA APPLICATION ON STUDENTS’ ACADEMIC AGHIEVEMENT IN 
ISSN 1648-3898 | /print/ yale a at aa pele 
ISSN 2538-7 1 38 /Online/ (PpP.1074-1088) 


Muema, J. S., Mulwa, D. M., & Mailu, S. N. (2018). Relationship between teaching method and students’ performance in 
Mathematics in public secondary schools in Dadaab sub-County, Garissa County, Kenya. [OSR Journal of Research & 
Method in Education, 8(5), 59-63. 

Mukherjee, S. (2018). Roles of multimedia in education. Edelweiss Applied Science and Technology, 2(1), 245-247. 
https://doi.org/10.33805/2576-8484.149 

Nbina, J. B. (2012). Teachers’ competence and students’ academic performance in senior secondary schools Chemistry: Is there 
any relationship? Global Journal of Educational Research, 11(1), 15-18. 

Neo, M., & Neo, K. T. K. (2001). Innovative teaching: Using multimedia in a problem-based learning environment. Education 
Technology & Society, 4(4), 19-31. 

Nworgu, B. G. (2003). Educational measurement and evaluation: Theory and practice. University Trust. 

Okeke, O. J. (2018). Effect of gender on the mean achievement score of chemistry students taught using Mend Mapping 
Teaching Strategy (MMTS) in Enugu. International Journal in Physical and Applied Sciences, 5(8), 32-50. 

Okwuduba, E. N., & Okigbo, E. C. (2018). Effect of teaching methods on student’s academic performance in Chemistry in 
Nigeria: Meta-analytical review. Bulgarian Journal of Science and Education Policy, 12(2), 418-434. 

Oladosu, C. T., Ayoola, R. A., Olokede, N. O., & Ojo, A. O. (2022). Teachers’ professional development, teachers’ quality and 
students’ academic achievements in the public-private partnership schools. Current Research in Language, Literature 
and Education, 4(9), 93-105. 

Organisation for Economic Co-operation and Development, (2018). The future of education and skills education 2030. 
https://www.oecd.org/education/2030/E2030%20Position%20Paper%20(05.04.2018).pdf 

Osinem, E. C. (2008). Managing agricultural education and training: Resources, principles and methods. Belony. 

Patel, C. (2013). Use of multimedia technology in teaching and learning communication skills: An analysis. International Journal 
of Advancement in Research & Technology, 2(7), 116-123. 

Podungge, R., Rahayu, M., Setiawan, M., & Sudiro, A. (2019). Teacher competence and student academic achievement. Advances 
in Economics, Business and Management Research, 144, 69-74. https://doi.org/10.2991/aebmr.k.200606.011 

Pooja, G. (2017). Study of the effect of teaching method on the academic achievement of school-going children of the semi- 
urban area, S school of Lucknow City. International Journal of Home Science, 3(2), 447-453. 

Ramnarain, U., & Hlatswayo, M. (2018). Teacher beliefs and attitude about inquiry-based learning in a rural school district in 
South Africa. South African Journal of Education, 38(1), 1-10. https://doi.org/10.15700/saje.v38n1a1431 

Safo-Adu, G., Quansah, R. E., & Ofori-Appiah, C. (2020). Appraisal of biology teachers’ practices in organising practical work in 
second cycle institutions in Eastern Region, Ghana. /nternational Journal of Academic Research and Reflection, 8(1), 1-16. 

Saida, H., & Mustapha, F. (2018). The effect of gender on university student school performance: The case of the 
National School of Agriculture in Meknes, Morocco. Kultura ir Visuomené. Socialiniy tyrimy Zurnalas, 9(1), 67-78. 
https://doi.org/10.7220/2335-8777.9.1.4 

Sarowardy, M. H., & Halder, D. P. (2019). The issues and challenges of using multimedia at a district level, specialized Girls’ 
College in Bangladesh. Creative Education, 10, 1507-1524. https://doi.org/10.4236/ce.2019.107110 

Shirani, B. N., Nasr-Isfahani, A.R., Rouhollahi, A., & Khalili, R. (2016). Effectiveness of teaching methods in higher education: 
Requirements and barriers. Journal of Advances in Medical Education & Professionalism, 4(4), 170-178. 

Sukma, D. M., Nurkamto, J., & Drajati, N. A. (2018). Multimedia-based presentation: Exploring interactivity emergence in the 
academic speaking classroom. Science, Engineering, Education and Development Studies (SEEDS): Conference Series, 2(1), 
29-40. https://doi.org/10.20961/seeds.v2i1.24250 

Thom-otuya, B. E.N., &lnko-tariah, D. C. (2016). Quality education for national development: The Nigerian Experience. African 
Educational Research Journal, 4(3), 101-108. 

Umar, G., Yagana, W. S. B., Kaltum, A., & Mohammed, W. B. (2015). Gender differences in students’ academic performance 
in Colleges of Education in Borno State, Nigeria: Implications for counselling. Journal of Education and Practice, 6(32), 
107-114. 

WAEC Chief Examiner’ Report, (2017), (2018) & 2019. West African Examination Council, Lagos, Nigeria. 

Zhang, Z. (2016). The use of multimedia in English teaching. US-China Foreign Language, 14(3), 182-189. 
https://doi.org/10.17265/1539-8080/2016.03.002 

Zudonu, O.C., & Njoku, Z. C. (2018). Effect of laboratory instructional methods on students’ attitude in some chemistry concepts 
at the senior secondary school level. Global Scientific Journals, 6(7), 46-61. 


Received: August 01, 2022 Revised: October 02, 2023 Accepted: November 04, 2023 


1087 


https://doi.org/10.33225/jbse/23.22.1074 QT 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


EFFECTS OF MULTIMEDIA APPLICATION ON STUDENTS’ ACADEMIC ACHIEVEMENT IN ISSN 1648-3898 /Print/ 
AGRICULTURAL SCIENCE 
(pP.1074-1088) ISSN 2538-7138 soniines 


Cite as: Ugwuoke, C. U., Babajide, E. A., Ekenta, L. U., Eze, G. E., Nwankwo, C. U., Ifeanyieze, F. O., Nwachukwu, C. U., Odoh, V. O., 
& Isiwu, E. C. (2023). Effects of multimedia application on students’ academic achievement in agricultural science. Journal of 
Baltic Science Education, 22(6), 1074--1088. https://doi.org/10.33225/jbse/23.22.1074 


Cajethan U. Ugwuoke PhD, Associate Professor, Department of Agricultural Education, 
University of Nigeria, Nsukka, Nigeria. 
E- mail: cajethan.ugwuoke@unn.edu.ng 
ORCID: https://orcid.org/0000-0002-6666-6660 


Emmanuel A. Babajide M.Sc., Lecturer, Department of Agricultural Education, University of 
Nigeria, Nsukka, Nigeria. 
E-mail: emmanuel.adebayo@unn.edu.ng 
ORCID: https://orcid.org/0000-0003-2424-6200 


Lilian U. Ekenta PhD, Senior Lecturer, Department of Agricultural Education, 
University of Nigeria, Nsukka, Nigeria. 
E-mail: lilian.ekenta@unn.edu.ng 
ORCID: https://orcid.org/0000-0001-5631-746x 


Godwin E. Eze PhD, Senior Lecturer, Department of Agricultural Education, University 
(Corresponding author) of Nigeria, Nsukka, Nigeria. 
E-mail: godwin.emeka@unn.edu.ng 
ORCID: https://orcid.org/0000-0003-3626-1678 


Clara U. Nwankwo PhD, Senior Lecturer, Department of Agricultural Education, University 
of Nigeria, Nsukka, Nigeria. 
E-mail: claranwankwo@unn.edu.ng 
ORCID: https://orcid.org/0000-0002-8453-1380 


Florence O. Ifeanyieze PhD, Associate Professor, Senior Lecturer, Department of Agricultural 
Education, University of Nigeria, Nsukka, Nigeria. 
E-mail: Florence.ifeanyieze@unn.edu.ng 
ORCID: https://orcid.org/0000-0002-9373-0271 


Christiana U. Nwachukwu PhD, Senior Lecturer, Senior Lecturer, Department of Agricultural 
Education, University of Nigeria, Nsukka, Nigeria. 
E-mail: christiana.nwachukwu@unn.edu.ng 
ORCID: https://orcid.org/0000-0002-6795-4410 


Vincent O. Odoh PhD, Senior Lecturer, Department of Mass Communication, Alex 
Ekwueme Federal University, Ndufu-Alike Ebonyi State, Nigeria. 
E-mail: ifnotvin@yahoo.com 


Edward C. Isiwu PhD, Senior Lecturer, Department of Agricultural and Home 
Economics Education, Michael Okpara University of Agriculture, 
Umudike, Nigeria. 
E-mail: edisiwuS50@yahoo.com 


1088 


Ass https://doi.org/10.33225/jbse/23.22.1074 


@® 6) This is an open access article under the 
(c) Creative Commons Attribution 4.0 
BY NC 


International License 


THE CONTEXT CONDITIONS 
STUDENTS’ REPRESENTATIONS 
OF THE HUMAN NUTRITION 
MODEL 


Teresa Zamalloa, 
Araitz Uskola, 
Ainara Achurra 


Introduction 


To understand human nutrition favors a global understanding of the 
human body itself. Nutrition is an essential function with the purpose of 
providing the organism with the nutrients and energy necessary to maintain 
its integrity. This involves transport and exchange of substances in dynamic 
connections between different systems, which are difficult to understand 
(Hmelo-Silver & Azevedo, 2006). Among the attempts to achieve systems 
thinking in human nutrition, it is worth highlighting the Components- 
Mechanisms-Phenomena (CMP) framework by Hmelo-Silver et al. (2017), 
which includes elements of the system (Components), the interrelated pro- 
cesses (Mechanisms) occurring among them, and the resulting outcomes 
or manifestations (Phenomena) of these processes within the system. The 
main obstacles to understanding nutrition can be sorted according to the 
CMP framework. Previous studies have described that it is frequent to restrict 
human nutrition to the digestive and respiratory organs (Components) (i.e., 
Reiss & Tunnicliffe, 2001). 

Other studies have concluded that although students generally pos- 
sess knowledge about the organs in the human body, they often struggle to 
place them within systems, establish connections between these organs, and 
gain a complete understanding of their respective functions (Mechanisms) 
(Aydin, 2016; Cuthbert, 2000; Ozsevgec, 2007; Reiss et al., 2002). Finally, Uskola 
et al. (2022) described that students did not explain the energy transfer of 
carbohydrates in the cells (Phenomena) when studying lactose intolerance. 

This could be explained because the teaching of the human body and 
its systems have traditionally been worked in a way that many students finish 
their primary and secondary education without developing an integrated 
and global notion about the human nutritional processes (Garcia-Barros et 
al., 2011; Granklint Enochson et al., 2015; Nunez & Banet, 1997), probably be- 
cause understanding the inter-relationships between concepts is harder than 


https://doi.org/10.33225/jbse/23.22.1089 


JOURNAL 
OF®BALTIC 
SCIENCE 
EDUCATION 


ISSN 1648-3898 pri 
ISSN 2538-71 38 jonine, 


Abstract. The human nutrition model is 
relevant in Biology education. Researchers 
and policymakers propose the introduction 
of modelling practices in science education, 
including the representation of the model. 
Despite being scarce, previous studies 

have shown that the guideline given to 
students conditions their performance 
when representing their models. This study 
addresses how the context given to 79 
preservice teachers (PSTs) in two cohorts at 
the end of a modelling sequence conditions 
the representations of the nutrition model 
constructed by them. The contexts were 

a child running and a lactose intolerant 
person. Written explanations and drawings 
of PSTs were analyzed according to the 
components-mechanisms-phenomena 
(CMP) framework. PSTs of both cohorts 
expressed a more developed nutrition 
model in a running context than inan 
intolerance context with respect to CMP 
aspects, which was shown by statistically 
significant differences. Therefore, the 
conclusion is that the context conditions 
the expression of the model. In this case, 

it was the context that appealed directly 

to the circulatory system and implied the 
use of energy, the one that led to a more 
complete representation of the human 
nutrition model. 


Keywords: evaluation context, preservice 
teachers, nutrition model, model 
representation 


Teresa Zamalloa, 
Araitz Uskola, 
Ainara Achurra 


University of the Basque Country, Spain 


1089 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


THE CONTEXT CONDITIONS STUDENTS’ REPRESENTATIONS OF THE HUMAN ISSN 1648-3898 /Print/ 
NUTRITION MODEL 
(PpP.1089-1102) ISSN 2538-7 1 38 /Online/ 


understanding the concepts separately (Lin & Hu, 2003). Thus, it seems to be necessary to overcome reductionist- 
teaching strategies based on the memorization of organ names when teaching the human body (Landinho et al., 
2022). Therefore, progress in constructing the nutrition mental model must be accompanied by the development 
of skills that allow its application in justifying phenomena and in personal actions and attitudes related to health 
or the environment (Garcia-Barros, 2016). 

This can be achieved by the scientific practice of modelling, which involves constructing, using, evaluating, 
and revising scientific models (Schwarz et al., 2009). A model can be defined as a representation of reality used to 
explain and predict scientific phenomena (Gilbert et al., 2000) and one of the phases of the modelling process is 
such representation of the mental model (Gilbert & Justi, 2016; Schwarz et al., 2009). As it is intentional, it can be 
communicative, cognitive or operational (Aduriz-Bravo, 2005). Thus, several studies have shown how representa- 
tions can be used to reflect on the model and learn science (Prain & Tytler, 2012; Tytler et al., 2020). 

In previous research on modelling, Uskola et al. (2022) suggested that research on how context facilitates the 
incorporation of process outcomes (Phenomena) was necessary. According to Gilbert (2006), the word context refers 
to a situation that provides meaning to words, phrases, and sentences. This is consistent with the idea that one of 
the goals of the use of context-based learning is to allow students to build mental maps that help them establish 
coherent connections and relationships of the scientific ideas that are being learned (Gilbert, 2006). Therefore, 
context-based learning in science education is a teaching strategy that promotes meaningful and socially relevant 
learning. The idea is that science should be taught based on real and meaningful situations for students, that are used 
as a focus, incorporating scientific concepts as they are necessary to better understand the situation (King, 2012). 

Asa proof of this, research has shown that how students activate ideas when they are asked a question depends 
on the context of that question (Hammer et al., 2005). In fact, students may show higher levels of performance in 
one context and lower levels in another context (Duncan et al., 2009). For example, Heredia et al. (2016) conducted 
a study on natural selection and found that students’ responses significantly differed depending on whether a 
plant or animal was used as the context. In particular, students chose the scientifically accepted response more 
frequently in an animal context than in a plant context, probably because students have been exposed to more 
animal examples in class or because they might not consider plants as living beings (Heredia et al., 2016). Thus, 
a lack of response to a particular question may be due to an inability to use existing knowledge in the context of 
evaluation (Nehm & Reilly, 2007). Concerning human body systems, it is well known that the mode of representing 
knowledge influences results; for example, the differences between written answers and drawings (Fancovicova 
& Prokop, 2019; Prokop & Fancovicova, 2006; Reinoso & Delgado-Iglesias, 2020; Uskola et al., 2022). However, the 
impact of the context has been comparatively less explored. Garcia-Barros et al. (2011) asked 342 children up to 
the age of 7 to draw the paths of food, drink, and air entering human and other animal bodies (dog, duck, fish). 
What they found was that they drew practically the same organs and structures in humans and animals, and it was 
only when they grew up that they began to differentiate, for example, fish organs. Khwaja and Saxton (2001) found 
that the instruction given to students for drawing conditioned the results: with the specific instruction “Draw the 
bones that are inside your body” students drew systems at higher levels than with the more general expression “Draw 
what you think is inside your body” used in a previous study. Based on these results, Prokop et al. (2009) compared 
the drawings of the urinary and endocrine systems made by children and university students after asking them 
to draw the inside of the human body and after asking them to draw a particular system. Although some of their 
results suggested that the drawings were similar to general and specific instruction, statistical analysis showed that 
they did not correlate with the endocrine system which they explained by the complexity of the endocrine system. 
Thus, they concluded that knowledge of the anatomy of the apparatus in question was a determining factor in 
drawing it well. The question of how a given context or guideline can influence the outcome, therefore, remains a 
subject of interest for research in science education. Teachers need to know more about how the guideline they 
give or the contexts they use to assess their students’ knowledge can condition such outcomes, so that they can 
design activities that allow students to express their knowledge in the best way. 


Research Aim and Research Questions 
This study addresses how the guideline given to students, in this case, the given context, conditions the rep- 
resentation of the nutrition model constructed by the students (in the present study preservice teachers (PSTs)). 


In that sense, the aim of the research was to study the differences between the representations in different con- 
texts and reflect on the characteristics of those contexts that can account for the differences, so that teachers can 
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consider them. Thus, the research question was: To what extent does the context condition the representation of 
PSTs’ human nutrition model at the end of a modelling sequence? 


Research Methodology 
General Background 


This study analyzed how the context given by the teacher conditioned the expression of the student’s model 
of human nutrition after participating in a modeling sequence (with the objective of constructing the model of 
human nutrition) in two cohorts of PSTs (2021/22 and 2022/23). Two contexts were used, one related to lactose 
intolerance (Context1-Ll) and the other to running (Context2-R). Categorization was made from a systemic view 
of human nutrition, according to the CMP framework proposed by Hmelo-Silver et al. (2017). Hence, the present 
study falls within the framework of mixed methods research (Creswell, 2012): the data collected are qualitative 
(drawings and written expressions) and then treated quantitatively for statistical analysis. 


Participants 


The participants were all the PSTs studying a Degree in Childhood Education (third year) at the Faculty of 
Education of Bilbao (Spain) in 2021/22 and 2022/23, that is 79 PSTs (32 in Cohort 1, 47 in Cohort 2). In both cohorts, 
the teacher was the first author. Purposeful sampling was undertaken. That is, researchers intentionally selected 
the participants that could help them understand the influence of the context in the expression of the mental 
models of nutrition (Creswell, 2012). For convenience participants were those willing and available to participate. 
All participants gave informed consent to participate in the research. It was clearly stated that participation was 
voluntary and anonymous. PSTs in the data collection process were coded as PST1-1 to PST1-32 in Cohort 1 and 
PST2-1 to PST2-47 in Cohort 2. The present study is part of the research project (M10_2021_161) in which this 
work was approved by the Ethics Commission for Research Involving Human Beings (CEISH) of the University of 
the Basque Country (UPV/EHU) on 20 May 2021). 


Teaching Sequence 


The sequence was structured in several stages as described before (Uskola et al., 2022). On a silhouette of a 
human body, participants first wrote and drew the elements and processes involved in lactose intolerance. After 
conducting research in small groups of three to five students, PSTs reconfigured their initial model under the guid- 
ance of the teacher. Once a consensus was reached, PSTs performed, in large groups (20-25 PSTs) and supervised 
by the teacher, a drama representation of nutrition, in which they drew a body silhouette (5 meters approximately) 
on the floor and acted as the elements (nutrients and gases) involved and the processes they undergo throughout 
the body. The drama activity was an experiential and scripted role-play with a structured frame, where participat- 
ing roles and the script were previously defined by the PSTs. The small groups then created a video in which the 
physical model was used to illustrate processes, representing their consensus model in a three-dimensional and 
dynamic physical model. Finally, PSTs were asked to draw and explain the elements and processes involved in two 
contexts. The representation modes were exactly the same to avoid differences coming from the representation 
mode (Fancovicova & Prokop, 2019; Prokop & Fancovicova, 2006; Reinoso & Delgado-lglesias, 2020; Uskola et al., 
2022). Context1-LI had the guideline “What happens in your body when you drink milk if you are intolerant? Draw 
and explain the elements and processes” that had been used in previous sequences over 4 years. As the PSTs did 
not express a complete model despite the improvements made over the years (Uskola et al., 2022), it was decided 
to include a new context to test if it was the context that showed the limitations. In Context2-R students were 
asked to answer the following: “You have run, and your heart rate has increased. Indicate what has happened in 
your body. Draw and explain the elements, pathways and processes that have occurred.” 


Data Collection and Analysis 


Data were gathered from students’ drawings and written texts at the end of the sequence, as recommended by 
Prokop and Fanéovicova (2006). Written explanations and drawings were analyzed according to the CMP framework 
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proposed by Hmelo-Silver et al. (2017), which was adapted to the task. The components could be macroscopic or 
microscopic. For the former components, five levels were established based on the number of boxes (digestive, 
respiratory, circulatory or excretory system and the organs (cells were also taken into account) where the energy 
is needed) and the number of connections between boxes mentioned (maximum 4 connections). A system was 
considered to be present even if the system was not completely represented. In the case of the microscopic com- 
ponents, the presence or absence of oxygen, carbon dioxide, nutrients and excretory products were considered. 
For the Mechanisms, the entrance and exit and the internal processes of microscopic components were evaluated. 
The internal processes reached values from 0 to 2 (2: nutrients absorption, gases diffusion or excretory products 
filtration and their transport; 1: nutrients absorption, gases diffusion or excretory products or their transport; 0: 
none). Phenomena are the outcome of the operating mechanisms (Snapir et al., 2017). In the present work concepts 
related to energy were included as phenomena. That is, mentioning energy throughout the explanation, consider- 
ing that nutrients are a source of energy and oxygen is involved in the production of energy, mentioning cellular 
respiration, addressing that energy is obtained in this process and mentioning the final use of energy. 

Figure 1 shows the aspects considered essential for the comprehension of human nutrition as a set of inte- 
grated systems, elements and processes (adapted from Nunez & Banet, 1997). It could be considered a schematic 
representation of the target scientific model PSTs should reach. 


Figure 1 
Schematic Representation of the Human Nutrition Scientific Model 
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Measures of central tendency (mean and median), dispersion (standard deviation) and frequency distribu- 
tion were calculated for all the variables in the two cohorts with the objective of comparing the runner context 
with the lactose-intolerant context. Wilcoxon signed-rank tests were conducted to statistically test the differences 
between both contexts; Hodges-Lehmann confidence intervals were also calculated. The effect size was measured 
with Cohen's d values, which were calculated for significant pairwise comparisons (p < .05) from z values obtained 
in Wilcoxon hypothesis tests (Ellis, 2010). Before tests were performed, normality was checked by Q-Q plots. All 
analyses were performed in SPSS and the Psychometrica calculator was used for the effect size analysis (Lenhard 
& Lenhard, 2016). Statistical analyses were undertaken and reinforced with qualitative data. 


Research Results 
Figure 2 shows the absolute frequencies for all the variables in Cohort 1. As shown in this figure, PSTs expressed 


amore developed model in Context2-R than in Context1-Ll in respect to the three evaluated aspects: components, 
mechanisms and phenomena. In Context2-R, PSTs represented more boxes and connections between boxes than 
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in Context1-Ll. That is, in Cohort 1, the basic structure of the nutrition model (digestive, respiratory, circulatory and 
excretory systems and organs/cells as well as pathways among them) was better expressed in Context2-R than 
in Context1-Ll. In Context1-Ll, components and mechanisms related to the nutrients were almost the only ones 
represented: presence of nutrients, entrance of nutrients to the digestive system and absorption and transport of 
nutrients. That is, representations of components and mechanisms involved in other systems than the digestive 
system were almost absent. In contrast, more than half of the PSTs in Context2-R mentioned components and 
mechanisms related to oxygen and carbon dioxide: oxygen, its entrance into the respiratory system and processes 
(diffusion and transport); and carbon dioxide and its exit from the respiratory system. Energy (phenomena) was 
completely absent in Context1-Ll. 


Figure 2 
Side by Side Comparison Bar Graph of Results in Cohort 1 


Cohort 1 


Context2-R Context1-Ll 


components 


mechanisms 


phenomena 


40 30 20 10 0 10 20 30 40 


Number of PSTs 


Note: Absolute frequencies for variables in Cohort 1 for Context2-R (left) versus Context1-Ll (right). Higher color 
intensity indicates higher value in the given variable (on the right). Note that only frequencies above 0 are shown; 
that is, only PSTs with some knowledge are shown. Variables are sorted in 3 categories (Components, mechanisms 
and phenomena). C1: number of boxes; C2: number of connections; C3: oxygen; C4: carbon dioxide; C5: nutrients; 
C6: excretory products; M1: entrance of oxygen; M2: entrance of nutrients; M3: exit of carbon dioxide; M4: exit 
of excretory products; M5: diffusion and transport of oxygen; M6: diffusion and transport of carbon dioxide; M7: 
absorption and transport of nutrients; M8: filtration and transport of excretory products; P1: energy; P2: nutrients 
as source of energy; P3: oxygen for production of energy; P4: cellular respiration; P5: obtain energy; P6: use energy. 
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In order to better define results in Cohort 1, examples of drawings of two PSTs of Cohort 1 are given (Figure 
3). Drawings made by PST1-32 show that this PST draws both respiratory and circulatory systems (as well as diffu- 
sion and transport) leaving out the digestive system in Context2-R, while in Context1-Ll the drawing is focused on 
the digestive system (Figures 3a and 3b). No phenomena are drawn nor described by PST1-32. Drawings made by 
PST1-5 show similar results (Figures 3c and 3d). 


Figure 3 
Examples of Drawings Made by PST 1-32 (a, b) and PST1-5 (c, d). 


Wn : RK \ | WN 


Table 1 includes measurements of the mean, median and standard deviation for both contexts in Cohort 1. 
The results given in Table 1 indicate that the differences between the means and medians obtained by Cohort 1 
in Context1-Ll and Context2-R are large in most variables. The scores obtained from Context1-Ll and Context2-R 
of Cohort 1 were compared through the Wilcoxon signed-rank test. Hodges-Lehmann confidence intervals were 
also calculated. As the table shows, the differences between Context1-L!l and Context2-R are significant (p < .05) 
for most of the variables. Exceptions are those variables related to the excretory system (the number of PSTs that 
represented them is 0 or close to 0 in any of the contexts); nutrient absorption and transport (means are 0.38 for 
Context1-LI and 0.34 for Context2-R) and cellular respiration (values close to 0). Results are reinforced by high 
Cohen’s d values (> .6 according to Hattie 2008). Thus, based on the confidence interval and Cohen’s d values, it is 
safe to say that PSTs in Cohort 1 performed better in Context2-R than in Context1-Ll. 


Table 1 
The Statistics Results in Cohort 1 


Variable Context M Mdn SD p cl% Cllimits | d Cohen 
itaved 1-1 1.47 1 0.84 < 001 95 15/-0.5 3.17 
(0,1,2,3,4,5) 2R 3 3 0.44 
Aiccnnceibne 1-1 0.38 0 071 < 001 95 A5I-A.0 2.58 
(0,1,2,3,4) 2-R 1.62 2 0.75 
- 1-1 0.03 0 0.18 < 001 95 1.0/-1.0 5.29 
(0,1) 2-R 0.91 1 0.47 
Gn 1-1 0 0 0 < 001 95 1.0/-0.5 2.97 
(0,1) 2R 0.69 { 0.47 
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shane 14 0.97 1 018 — <.001 9 = Osi.0 = 2.42 
(0,1) 2R 0.38 0 0.49 

1-Ll 0 0 0 z _ 
Excretory products (0,1) 

2-R 0 0 0 
Entrance of 02 1-Ll 0 0 0 < .001 95 1.0/1.0 4.65 
(0.1) 2R 0.84 1 0.37 
Entrance of nutrients 1-1 0.94 ‘ 0.25 < 001 95 -1.0/-1.0 6.22 
(0.1) 2R 03 0 0.18 
Exit of CO2 1-Ll 0 0 0 < 001 95 0.5/1.0 2.76 
(0,1) 2R 0.66 1 0.48 
Exit of excretory prod- tL 0 0 0 0.32 - - 
ces (01) 2R 0.03 0 0.18 
02 processes 1-Ll 0 0 0 < .001 95 -1.5/-1.0 2.97 
(0,1,2) 2R 1.34 15 0.74 
CO2 processes tL 0 0 0 0.014 : 3 0.97 
(0,1,2) 2R 0.37 0 0.75 
Nutrients processes 1-1 0.38 0 0.7 0.782 
(01,2) 2R 0.34 0 0.60 
Excretory products tL 0 0 0 - 
processes (0,1,2) oR 0 0 0 
Energy 1-L 0 0 0 < .001 90 -0.5/-0.5 1.76 
(0.1) 2R 0.44 0 05 
Nutrient as source of tL 0 0 0 0.046 - - 0.76 
energy (0,1) 2R 0.12 0 0.34 
02 for production of tL 0 0 0 0.005 - - 1.15 
anergy (0,1) 2R 0.25 0 0.44 
Obtain energy tL 0 0 0 0.008 - - 1.06 
(0,1) 2R 0.22 0 0.42 

1-L 0 0 0 0.157 
Cellular respiration (0,1) 

2-R 0.06 0 0.25 
Use energy tL 0 0 0 0.002 : : 1.35 
(0.1) 2R 031 0 0.47 


Note: Descriptive statistics, p values for the Wilcoxon signed-rank tests, confidence intervals for the median differences (Hodges 
and Lehmann) with upper and lower limits and Cohen’s d values for results in Cohort 1 (n=32). 


In respect to Cohort 2, the results are congruent with those in Cohort 1; that is, PSTs in Cohort 2 also performed 
better in Context2-R than in Context1-Ll. Figure 4 shows the absolute frequencies for all the variables in Cohort 
2. Again, PSTs represented a more developed structure in Context2-R than in Context1-Ll, in terms of boxes and 
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connections among them. In Context1-Ll, components and mechanisms related to the nutrients were the most 
frequently represented: presence of nutrients, entrance of nutrients to the digestive system and absorption and 
transport of nutrients. In contrast, PSTs in Context2-R mentioned higher frequency components and mechanisms 
related to all systems integrated in the nutrition model. Variables related to energy (phenomena) were also more 
frequent in Context2-R than in Context1-Ll. 


Figure 4 
Side by Side Comparison Bar Graph of Results in Cohort 2 
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Note: Absolute frequencies for variables in Cohort 2 for Context2-R (left) versus Context1-Ll (right). Higher color 
intensity indicates higher value in the given variable (on the right). Note that only frequencies above 0 are shown; 
that is, only PSTs with some knowledge are shown. Variables are sorted in 3 categories (components, mechanisms 
and phenomena). C1: number of boxes; C2: number of connections; C3: oxygen; C4: carbon dioxide; C5: nutrients; 
C6: excretory products; M1: entrance of oxygen; M2: entrance of nutrients; M3: exit of carbon dioxide; M4: exit 
of excretory products; M5: diffusion and transport of oxygen; M6: diffusion and transport of carbon dioxide; M7: 
absorption and transport of nutrients; M8: filtration and transport of excretory products; P1: energy; P2: nutrients 
as source of energy; P3: oxygen for production of energy; P4: cellular respiration; P5: obtain energy; P6: use energy 
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Figure 5 shows examples of drawings of two PSTs in Cohort 2. Drawings made by PST2-10 in Context2-R reached 
the highest level in the target human nutrition model (Figure 1) in terms of components, while in Context1-Ll the 
drawing is mainly focused on the digestive system (Figures 5a and 5b). Drawings made by PST2-16 show similar 
results but, in this case, the explanations given in Context2-R described the mechanisms and phenomena involved 
(Figures 5c and 5d). 


Figure 5 
Examples of Drawings Made by PST2-10 (a, b) and PST2-16 (c. d) 


Table 2 includes values for the mean, median and standard deviation for both contexts in Cohort 2. As in Cohort 
1, differences between the means and medians between Context1-Ll and Context2-R are large in most variables. 
In respect to p values derived from Wilcoxon signed-rank tests, differences between Context1-Ll and Context2-R 
are significant (p < .05) for all the variables except for the processes related to nutrients (mean values are 0.62 for 
Context1-Ll and 0.82 for Context2-R). Results are reinforced by high Cohen's d values (> 0.6 according to Hattie 
(2008)). Thus, as in Cohort 1, based on the confidence interval and Cohen’s d values, it is safe to say that PSTs in 
Cohort 2 performed better in Context2-R than in Context1-Ll. Therefore, based on the results of Cohort 1 and Co- 
hort 2, the Context2-R was a significantly better context for the representation of PSTs’ models of human nutrition. 


Table 2 
The Statistics Results in Cohort 2 


Variable Context M Mdn SD p cl% Cl limits d Cohen 
1-Ll 3.38 3 1.29 < 001 95 -2.0/-1.0 1.22 
N boxes 
(0,1,2,3,4,5) 2-R 4.33 5 1.01 
F 1-Ll 1.36 1 1.61 < 001 95 -2.5/-1.0 1.83 
N connections 
(0,1,2,3.4) 2-R 3.10 4 1.18 
02 1-Ll 0.15 0 0.36 < 001 95 -1.0/-0.5 3.23 
(0.1) 2-R 0.88 1 0.33 
C02 1-Ll 0.11 0 0.31 < 001 95 -1.0/-0.5 3.06 
(0,1) 2-R 0.79 1 0.41 
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Variable Context M Mdn SD p cl% Cl limits d Cohen 
NGRiGAS 1-LI 1 1 0 0.008 - - 0.84 
(0.1) 2-R 0.84 1 0.37 
1-LI 0.85 0 0.28 < 001 95 -0.5-0.5 2.05 
Excretory products (0,1) 
2-R 0.63 1 0.49 
Cafaneerck 02 1-LI 0 0 0 < 001 95 1.0/-0.5 3.62 
(0,1) 2-R 0.78 1 0.42 
1-LI 0.85 1 0.36 <.001 95 0.5/0.5 1.45 
Entrance of nutrients (0,1) 
2-R 0.37 0 0.49 
Exit of CO2 1-LI 0.08 0 0.28 <.001 95 -1/-0.5 2.24 
(0,1) 2-R 0.67 1 0.47 
1-LI 0.08 0 0.28 <.001 90 -0.5/-0.5 
Exit of excretory products (0,1) 
2-R 0.49 0 0.26 
1-LI 0.15 0 0.51 < 001 95 -2/-1 2.98 
O2 processes 
(0.1.2) 2-R 1.55 2 0.79 
CO2 processes 1-LI 0.11 0 0.43 0.014 95 -1.5/-1 2.34 
(0.1.2) 2-R 1.27 2 0.95 
1-LI 0.62 1 0.68 0.158 - - 
Nutrients processes (0,1,2) 
2-R 0.82 1 0.85 
1-LI 0.11 0 0.43 < 001 95 -1/-0.5 1.44 
Excretory products processes 
(0.1.2) 2-R 0.89 0 0.96 
1-LI 0.13 0 0.34 < 001 95 -1/-0.5 2.24 
Energy 
(0,1) 2-R 0.71 1 0.46 
: 1-LI 0.04 0 0.20 < 001 95 -0.5/-0.5 1.72 
Nutrient as source of energy 
(0.1) 2-R 0.45 0 0.50 
02 for production of energy tH ie v om Pe : : ie 
(0.1) 2-R 0.31 0 0.47 
: 1-LI 0.13 0 0.38 < 001 90 -0.5/-0.5 1.33 
Obtain energy 
(0.1) 2-R 0.51 1 0.50 
1-LI 0.04 0 0.20 < 001 95 -0.5/-0.5 1.79 
Cellular respiration (0,1) 
2-R 0.49 0 0.50 
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Variable Context M Mdn SD p cl% Cl limits d Cohen 
1-LI 0.04 0 0.20 < .001 80 -0.5/-0.5 1.45 
Use energy 
(0.1) 2-R 0.42 0 0.49 


Note: Descriptive statistics, p values for the Wilcoxon signed-rank tests, confidence intervals for the median differences (Hodges 
and Lehmann) with upper and lower limits and Cohen’s d values for results in Cohort 1 (n=47). 


Discussion 


This study aimed to address how the context conditioned PSTs’ representation of their human nutrition 
model after participating in a modeling sequence whose objective was to construct the human nutrition model. 

The findings showed two noteworthy aspects that deserve to be discussed. Firstly, in response to the research 
question, the PSTs showed a more complete model in Context2-R than in Context1-Ll. This was evident in terms 
of representing a greater number of elements, organs and systems involved in human nutrition, more processes 
and amore thorough consideration of the underlying mechanisms. Secondly, this difference between the contexts 
persisted across both cohorts, even though the results were better in Cohort 2. 

The results in Cohort 2 were better than those in Cohort 1 in both contexts. However, in Cohort 2, non-digestive 
system aspects still rarely appeared in Context1-Ll. The situation present in quite a few PSTs was that, while in Co- 
hort 1 Context1-Ll was limited to the digestive system and Context2-R to the circulatory and respiratory systems 
(Figures 2 and 3), in Cohort 2, Context2-R reflected also the digestive and urinary systems, the various elements, 
processes, and energy extraction, while Context1-Ll included some organs of the other systems but did not expand 
the elements, processes, and energy extraction to the same extent (Figures 4 and 5). This indicates that the model 
represented in Context2-R in Cohort 2 was in many cases quite complete and close to the target model represented 
in Figure 1, but such an improvement was not reflected to the same extent in Context1-Ll. In fact, the difference 
between the contexts was statistically greater in Cohort 2. 

This reinforces the idea that Context2-R was more appropriate for representing the model of nutrition and that 
Context1-Ll, on the contrary, has been limiting. In fact, it did not reflect the development in the model that PSTs 
had made. This finding supports previous research indicating that instructions and contexts have varying limita- 
tions and potentials and should be taken into account when designing activities to assess students’ knowledge 
(Heredia et al., 2016; Khwaja & Saxton, 2001; Nehm & Reilly, 2007; Prokop et al., 2009). 

The emerging question from this study is what limitations Context1-Ll has and/or where the potential of 
Context2-R lies. For this new question, hypotheses from researchers, which would need to be tested in future 
research, are that the context facilitates the representation of the nutrition model when it refers to the circulatory 
system and when it represents an energy use phenomenon. 

Regarding the first hypothesis, the circulatory system is characterized by connecting the different systems, 
as shown in Figure 1. Its function is to transport elements such as nutrients, gases, excretory products and other 
substances within the organism; these elements come and go from different systems. This could potentially facili- 
tate students’ understanding of interrelationships and transport processes. In fact, Tripto et al. (2017) introduced 
the circulatory system as a mediating system, that links the function of the body’s other systems and participates 
in the activities that enable homeostasis. 

Concerning the second hypothesis, in Context1-LI the aim was for PSTs to allude not only to how the intolerance 
is produced but also to the consequences it has both on the formation of symptoms and on the failure to obtain 
energy due to the non-absorption of lactose in the small intestine, i.e., to the phenomenon produced (Hmelo-Silver 
et al., 2017; Uskola et al., 2022). However, the PSTs focused on explaining the phenomenon as a digestive problem. 
This aligns with Franco and Colinvaux’s (2000) idea of synthesis when representing mental models due to the 
economy of cognitive resources. A representation is never a complete reproduction of the mental model because it 
implies the selection of what aspects will be represented and what other aspects will be left out of the representa- 
tion (Prain & Tytler, 2012; Tytler et al., 2020). The idea here is that, depending on the context, PSTs’ representations 
of their models were more or less synthetic. In Context2-R the phenomenon focused on the use of energy for run- 
ning. In fact, energy was mentioned by 44% in Cohort 1 and 72% in Cohort 2, while only 0% and 13% respectively 
in the case of Context1-Ll. Explaining where this energy comes from requires PSTs to focus on both nutrients and 
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oxygen, their pathways and processes, and the disposal of the excretory products produced by obtaining usable 
energy in the cells. Therefore, teachers should choose a phenomenon that necessarily requires students to make 
reference to and use their knowledge of the organs, elements, structures, pathways and processes to be assessed. 

A potential way to test the weight of these hypotheses could involve using a context in which only one of 
the two factors is present. For example, a teacher might adjust the intolerance scenario to incorporate an energy- 
related aspect, such as indicating that the intolerant person experiences fatigue. Conversely, a teacher could use 
a context only focused on the circulatory system, such as the one where students have to predict outcomes fol- 
lowing blood loss from a wound. 

Other studies on human systems have used instructions focused on the structure (in its general or specific 
form (Prokop et al., 2009; Reinoso & Delgado-Iglesias, 2020; Reiss & Tunniclife, 2001), others in the paths several 
elements follow inside the body (Garcia-Barros et al., 2011; Robles-Moral et al., 2023). The results of the study sug- 
gest that, when teaching the nutrition topic, guidelines for students and contexts containing references to the 
circulatory system and energy use, that is, those that emphasize interrelationships between the components of 
the model and the phenomenon, should be used to evaluate the human nutrition model. 


Limitations 


The sample selection was non-random, limiting the capacity to draw statistically significant conclusions 
between contexts. Nevertheless, the congruence of results across both cohorts was noteworthy for this study’s 
purpose. PSTs demonstrated improved performance within the runner context in both Cohort 1 and Cohort 2. 


Conclusions and Implications 


The present study showed that the context conditions the expression of the mental model. Given that teachers 
and their proposed activities constitute one of the main components in the teaching-learning process, this study 
added some evidence on the importance of selecting an appropriate context for the students to express their 
mental models. Teachers play a crucial role in offering the most appropriate context to evaluate the knowledge 
students construct. The findings of the present study showed that the guidelines given to students did condition 
the representation of the mental model of the students. The results lead to the suggestion that guidelines given 
to students and contexts that appeal to the interrelationships between the components of the model and the 
phenomena resulting from them should be used to assess the nutrition model. Contexts like these could be used 
not only as assessment tools but also for their construction in modelling sequences that involve representing the 
model to be assessed and revised. Representing the model gives the opportunity not only to communicate what 
has been learned but also to be able to ask new questions, reflect and construct knowledge. Incorporating contexts 
that facilitate the model to be represented as fully as possible can stimulate more questions, broader evaluation 
and, therefore, the construction of models closer to scientific ones. 
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Introduction 


Over the years, formative assessment has become one of the research 
topics in science education. In particular, the increasing number of pub- 
lications on formative assessment since the early 2000s has shown how 
important this topic is for science educators and teachers (Kariri et al., 
2018). Formative assessment is essential for learning and teaching because 
it provides continuous feedback and information on student comprehen- 
sion, progress, and needs (Furtak & Ruiz-Primo, 2008; Kariri et al., 2022; 
Wiliam, 2009). Unlike summative assessments, typically used to evaluate 
students at the end of a unit or course, formative assessments are designed 
to be embedded in classroom learning processes (Cisterna & Gotwals, 2018; 
Kariri et al., 2022; Menéndez et al., 2019). Hence, it is crucial to learning and 
teaching because it facilitates student development, guides instruction, 
and fosters a deeper understanding of content (Clark, 2011; Menéndez et 
al., 2019). It empowers students to take responsibility for their learning and 
fosters a collaborative and reflective learning environment (Jacoby et al., 
2013; Taras, 2002). Formative assessments allow students to receive timely 
feedback on their learning that helps them understand their strengths and 
areas for improvement (Havnes et al., 2012; Kariri et al., 2022; McCarthy, 2017). 
This feedback allows students to adjust their responsibility during learning, 
leading to increased motivation and engagement (Cauley & McMillan, 2010; 
Leenknecht et al., 2020). 

The research literature demonstrates that formative assessment signifi- 
cantly affects students’ learning (Hidayat & Irdiyansyah, 2023; Ozan & Kincal, 
2018). Evaluation is crucial to effective learning (Menéndez et al., 2019). 
Formative assessments are characterized primarily by their purpose and tim- 
ing in identifying and discussing a student’s performance and determining 
the required next steps. Studies have shown that it helps students achieve 
higher academic objectives and improves curriculum and teaching practices 
(Kariri et al., 2022). With its importance for learning and teaching, formative 
assessment in science education has been a topic of great research interest 
to improve teaching and learning experiences in the education field (Black, 
2009; Park, 2019). 


https://doi.org/10.33225/jbse/23.22.1103 


JOURNAL 
OF®BALTIC 
SCIENCE 
EDUCATION 


ISSN 1648-3898 rin 
ISSN 2538-71 38 jonin, 


Abstract. Although the number of 
publications on formative assessment in 
science education has long been high, 
there is no bibliometric analysis or scientific 
mapping in the literature to understand 
research trends in formative assessment 
research. This research aimed to examine 
the bibliometric results of articles on 
formative assessment in science education. 
Based on the criteria for inclusion and 
exclusion, 94 articles were selected for 
analysis. The results show that between 
2015-2016 and 2020-2022, the number 

of publications on formative assessment 
increased substantially. Among the top ten 
institutions that contributed to the research 
are three institutions from the United 
States. The results also show that the 
United States had the most publications. 
Analysis of the co-citations showed that the 
Journal of Research in Science Teaching, 
Science Education, and the International 
Journal of Science Education were cited 
more than 100 times. The three most cited 
studies were published in Assessment in 
Education: Principles, Policy, and Practice. 
The results show that the researcher 

used formative assessment, students, 
science education, teaching, education, 
engineering education, curricula, STEM, 
and e-learning in research on formative 
assessment. In light of the obtained results, 
practical suggestions for further studies are 
made in the conclusion. 


Keywords: formative assessment, science 
education, SCOPUS, bibliometric analysis 


Yulian Zhang 

Huaqiao University, China 

Weijun Wang 

No.5 Middle School of Quanzhou, China 
Yi Xian 


Sichuan University, China 

Xianfeng Wang 

Experimental Primary School of Mudan 
District, China 

Jiabin Huang 

No. 1 Middle School of Shishi, China 


1103 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


THE RESEARCH STATUS OF FORMATIVE ASSESSMENT IN SCIENCE EDUCATION ISSN 1648-3898 
(pP.1103-1119) 


/Print/ 


ISSN 2538-7138 soniines 


Researchers have explored and used various formative assessment strategies and techniques that can be used 
in science classrooms (Decristan et al., 2015; Park, 2019; Quellmalz et al., 2012). These strategies and techniques 
include concept mapping, self-assessment, peer assessment, and feedback strategies. Studies have examined the 
effectiveness of these strategies in promoting student engagement, conceptual understanding, and scientific 
inquiry skills (Decristan et al., 2015; Park, 2019; Quellmalz et al., 2012). 

Growing interest is in developing learning progressions and frameworks that outline students’ science under- 
standing of developmental stages (Alonzo, 2017; Furtak et al., 2018). Formative assessment is critical in creating 
these learning progressions by providing insights into student thinking and conceptual development over time 
(Alonzo, 2017; Furtak et al., 2018). With this aim, many studies have been conducted to assess the effects of forma- 
tive assessment on students’ learning in science education literature (Furtak et al., 2018; Hidayat & Irdiyansyah, 
2023; Leenknecht et al., 2020; Ozan & Kincal, 2018). To keep up with the latest research on formative assessment in 
science education, it is important to monitor the development of the knowledge produced in science education 
by searching the most recent academic journals, conference proceedings, and databases specializing in science 
education research. 

To assist students in learning and teaching, studies need to be reviewed that summarize the current state of 
research on formative assessment to inform researchers. However, no bibliometric analysis or scholarly mapping 
of formative assessment in the science education literature has been found. Only one research (Sudakova et al., 
2022) conducted a bibliometric analysis of online formative assessment in higher education. They collected the 
Scopus database and analyzed 898 publications. Their results demonstrated that the rate of increase between 
2019 and 2021 was greater than the collaboration index of 3.02 co-authors per document. The highest percentage 
of annual citations per document was 17.44%. The growth rate between 2019 and 2021 was greater. In addition, 
their results suggested that the publications frequently used online assessment, blended learning, e-learning, and 
COVID-19 keywords. To address these gaps, this study used a bibliometric analysis to thoroughly review forma- 
tive assessment in science education to identify trends in formative assessment research covering a longer period 
from 2008 to 2022. Researchers use bibliometric evaluation techniques to determine a single author’s influence 
and establish the relationship between two or more authors or works. Therefore, this study aimed to examine the 
research status of formative assessment in science education. 


Research Significance 


Researchers have agreed that bibliometric analysis enables researchers to identify and analyze trends and 
patterns in a specific research area and topic. By analyzing publication output, citation patterns, and collabora- 
tion networks, researchers can gain insight into the topics, methodologies, and theories that have garnered the 
most attention in a field. These insights assist researchers in comprehending the current state of knowledge and 
identifying possible research gaps. Thus, bibliometric analysis provides a quantitative evaluation of the impact and 
influence of research on formative assessment. Researchers can contribute to developing new research questions 
and priorities by identifying under-researched topics or aspects that require further examination. In addition, 
bibliometric analysis can reveal emerging fields or novel approaches in formative assessment research, allowing 
researchers to follow the most recent advancements. 

Bibliometric research provides evidence-based information to guide education policy, curriculum develop- 
ment, and instructional practice decisions. Policymakers and educators can use bibliometric analysis to comprehend 
the efficacy and impact of formative assessment strategies and make informed implementation decisions. In brief, 
bibliometric research on formative assessment is important for researchers because it enables them to gain insights 
into the field, evaluate research impact, identify research gaps, and contribute to evidence-based educational 
decision-making. This study focuses on articles published between 2008 and 2022 on formative assessment in 
science education. These articles were analyzed using bibliometric mapping regarding the most frequently used 
keywords, most frequently cited authors, and most frequently cited journals. 
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Research Methodology 
Background 


Bibliometric analysis allows researchers to analyze and evaluate themes, patterns, and connections between 
scientific papers in a discipline. This technique examines published data on authors, citations, institutions, journals, 
keywords, and subjects to create networked knowledge maps in a research area (van Eck et al., 2010). It allows 
researchers to use co-citation and keyword analysis to identify and examine research themes in publications about 
current research on formative assessment in science education. Therefore, this finding provides the opportunity 
to gain valuable insight into the current state of research on formative assessment. 


Data Collection 


In this study, bibliometric mapping analysis was used. To answer the research question, the researchers 
searched the database for publications on formative assessment in science education articles published between 
2008 and 2022. The SCOPUS database was selected as the primary source of literature. It is a top-level research 
platform that facilitates discovering, analyzing, and disseminating knowledge in the sciences, social sciences, arts, 
and humanities (Elsevier, 2023). It is used to find relevant research papers by indexing the most relevant journals 
in the field of education and providing the necessary data for bibliometric analysis. 

The time frame was restricted to the years 2008 to 2022. To ensure quality consistency, “English” was selected 
as the language, and all publication types were selected as the document type. The subject area keywords “forma- 
tive assessment” and “science education” were selected to collect the data. The “education/educational research” 
category was used to perform an advanced search, and researchers found 122 articles related to formative assess- 
ment in science education. The full texts of all articles were downloaded. Two researchers assessed each article 
using specific criteria for inclusion and exclusion. For inclusion, articles pertained to formative assessment and 
one of the science education disciplines, including biology, chemistry, physics, science, and STEM education. The 
second was whether the articles were conducted in the education field. For example, some articles were found 
outside of education fields and unrelated to one of the disciplines in science education specifically. Consequently, 
94 articles from numerous journals and proceedings were chosen for analysis in this study. Of the eligible articles 
for analysis, the number of articles was 63, while there were 26 conference papers, two book chapters, two reviews, 
and one book. Figure 1 is a summary of the article selection procedure for this study. 


Bibliometric Analysis 

SCOPUS was chosen as the literature source for the bibliometric mapping analysis using the same keywords. 
Since it was impossible to examine each article individually, keywords related to formative assessment and science 
education-related keywords were used. Using the advanced search function, “formative assessment” and “science 
education” or “biology education’, “chemistry education,’ “physics education, “STEM’, and “STEM education” were 
entered in the subject field in the database. The full records and references were then downloaded in Bibtext and 
CSV formats. These files were later converted to tab-delimited (Windows) file formats. The file was uploaded to 
the program VOSViewer. The bibliometric mapping analysis comprises articles published between 2008 and 2022. 

The VOSViewer program was used for the bibliometric analysis to generate a network representation of the 
articles’ most frequently used keywords, abstract words, citation analyses, and co-citation analyses. Two research- 
ers analyzed the data together. 


Research Results 
Change in the Number of Publications 
Figure 1 shows an annual change in the number of publications on formative assessment from 2008 to 2022. 
According to Figure 1, the number of publications on formative assessment increased importantly between 2015- 


2016 and 2020-2022. Namely, there was an increase in the number of formative assessment articles over the past 
three years. 
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Figure 1 
Change in the Number of Publications 
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Table 1 shows the institutions that contributed to formative assessment research between 2008 and 2022. 
The top ten institutions that contributed to the research on formative assessment were first the University of 
Colorado Boulder, the University of Nebraska-Lincoln (n = 7), the University of California, Berkeley (n = 4), the Uni- 
versity of Missouri (n = 3), the School of Natural Resources (n = 3), University of the Andes (n =3), North Carolina 
State University (n = 2), Educational Testing Service (n =2), and University of South Bohemia (n = 2). The results 
revealed that seven of the top ten institutions came from one country, the United States. According to this result, 
the institutions in the United States conducted much more research on formative assessment in science educa- 
tion during the last fifteen years. Four countries entered into the list among the top ten institutions. They are the 
United States, Colombia, the Czech Republic, and Switzerland. In particular, the University of Colorado Boulder 
and the University of Nebraska-Lincoln had contributed the most to research on formative assessment. These 
results indicated nine institutions, excluding Colombia, originate from developed nations. It is important to note 
that universities in developed nations, particularly the United States of America (USA), have been at the forefront 
of formative assessment research in the academic literature. 


Table 1 
Top 10 Institutions Contributed to Formative Assessment Research 


Institutions Country Number of publications 
University of Colorado Boulder United States 7 
University of Nebraska-Lincoln United States 7 
University of California, Berkeley United States 4 
University of Missouri United States 3 
University of the Andes Colombia 3 
North Carolina State University United States 2 
Educational Testing Service United States 2 
University of South Bohemia Czech Republic 2 
The University of Arizona United States 2 
FHNW University of Applied Sciences and Arts Switzerland 2 
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Figure 2 shows the top 10 countries that contributed to formative assessment research. The results in Figure 2 
demonstrate that the United States had the most publications (n= 45, or 47.8%). Germany and Israel rank second in 
productivity (n = 6, 6.3%). The Netherlands came third with four publications (n = 4, or 4.2%). In terms of the number 
of publications on attitudes and STEM, the following nations are represented: Colombia (n = 3, 3.2%), Denmark 
(n = 3, 3.2%), Greece (n =3, 3.2%), Spain (n = 3, 3.2%), Turkey (n = 3, 3.2%), and the United Kingdom (n =3, 3.2%). 


Figure 2 
Top 10 Countries That Contributed to Formative Assessment Research 
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Analysis of Sources 


Table 2 displays the top ten sources determined by co-citation analysis within the Vosviewer software. The 
journal with the most citations is the Journal of Research in Science Teaching (JRST), Science Education, and the 
International Journal of Science Education (IJSE) had more citations than 100 times. The scopes of these journals are 
closely related to formative assessment in science education. The impact factor for the JRST was 3.918%. In 2021, 
science education had a 6.0 impact factor. In 2021, the IJSE had a five-year impact factor of 2.77. According to the 
cluster analysis (see Figure 3), the journals that published articles on formative assessment were collected in two 
clusters. Accordingly, the first cluster included these journals: JRST, Science Education, Journal of Science Teacher 
Education, Educational Researcher, Assessment in Education: Principles, Policy & Practice, Applied Measurement in 
Education, and Teaching and Teacher Education. The second cluster incorporated the IJSE, Review of Educational 
Research, and Computers & Education journals. The main pattern of these journals is that prominent publishers 
have published them for more than 30 years. 


Table 2 
Top 10 Sources by Co-Citation Analysis 


Citations Total link Strength Cluster 
1 Journal of Research in Science Teaching 158 2346 1 
2 Science Education 105 1637 1 
3 International Journal of Science Education 103 1389 2 
4 Journal of Science Teacher Education 40 798 1 
5 Educational Researcher 42 745 1 
6 Assessment in Education: Principles, Policy & Practice 41 727 1 
7 Review of Educational Research 51 647 2 
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Citations Total link Strength Cluster 
8 Computers & Education 66 619 2 
9 Applied Measurement in Education 30 595 1 
10 Teaching and Teacher Education 31 535 1 


Figure 3 
Analysis of Sources by Co-Citation Analysis 
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Analysis of Authors 


Table 3 lists the ten most co-cited authors by co-citation analysis. Only Dylan Wiliam, Erin Marie Furtak, Paul 
Black, Maria Araceli Ruiz-Primo, and Richard J. Shavelson were cited more than fifty times. The remaining five authors 
were co-cited over thirty times, including Bronwen Cowie, Christine Harrison, Marcia C. Linn, Yue Yin, and Beverley 
Bell. Figure 4 shows an analysis of the most cited authors by co-citation. Based on the most cited author analysis, 
Table 3 and Figure 4 reveal that six out of the top ten authors are from the United States. Meanwhile, three authors 
come from New Zealand, and one is from the United Kingdom. 


Table 3 
Top 10 Authors by Co-Citation Analysis 


Total link 


Author Organization Country Citations Strength Cluster 
1 Dylan Wiliam Educational Testing Service United States 93 1470 2 
2 Erin Marie Furtak University of Colorado Boulder United States 82 1638 1 
3 Paul Black King’s College London United Kingdom 80 1269 2 
4 Maria Araceli Ruiz-Primo Stanford University United States 58 1261 1 


1108 


WSS” https://doi.org/10.33225/jbse/23.22.1103 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


ISSN 1648-3898 /Print/ THE RESEARCH STATUS OF FORMATIVE ASSESSMENT IN SCIENCE EDUCATION 
(PpP.1103-1119) 


ISSN 2538-7138 /oniine/ 


Pare ae Total link 
Author Organization Country Citations Strength Cluster 
5 Richard J. Shavelson Stanford University United States 52 1172 1 
6 Bronwen Cowie The University of Waikato New Zealand 42 857 1 
7 Christine Harrison The University of Waikato New Zealand 38 827 2 
8 Marcia C. Linn University of California Berkeley United States 36 82 2 
9 Yue Yin AS ee ens United States 35 842 1 
University of Hawaii, Manoa 
10 Beverley Bell The University of Waikato New Zealand 30 639 1 
Figure 4 
Analysis of Authors by Co-Citation Analysis 
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Table 4 displays the top ten references based on co-citation analysis. The first six references were cited by 
researchers more than five times. Three studies among the top ten references based on co-citation were from 
Assessment in Education: Principles, Policy & Practice journal. The other two studies were from the Journal of 
Research in Science Teaching. 


Table 4 
Top 10 References by Co-Citation Analysis 


Authors(s) Source Citations Tera\inks Cluster 
Strength 
1 Assessment and Classroom Learning Paul Black & Dylan Assessment in Education: 8 21 1 
(Black & Wiliam, 1998) Wiliam Principles, Policy & Practice 
2 The Power of Feedback (Hattie & Timper- John Hattie and Helen Review of Educational 7 27 2 
ley, 2007) Timperley Research 
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From Evidence to Action: A Seamless Pro- 
cess in Formative Assessment? (Heritage 
et al., 2009) 


Formative assessment: a critical review 
(Bennett, 2011) 


The characteristics of formative assessment 
in science education (Bell & Cowie, 2001) 


Developing the theory of formative assess- 
ment (Black & Wiliam, 2009) 


The missing disciplinary substance of 
formative assessment (Coffey et al., 
2011) 


Strategies and Challenges to Changing the 
Focus of Assessment and Instruction in Sci- 
ence Classrooms (Duschl & Gitomer, 1997) 


Exploring teachers’ informal formative 
assessment practices and students’ under- 
standing in the context of scientific inquiry 
(Ruiz-Primo & Furtak, 2007) 


A Model of Formative Assessment in Sci- 
ence Education (Cowie & Bell, 1999) 


Analysis of References 


Authors(s) 


Margaret Heritage, 
Jinok Kim, Terry Vend- 
linski, Joan Herman 


Randy Elliot Bennett 


Beverley Bell, Bron- 
wen Cowie 


Paul Black & Dylan 
Wiliam 


Janet E. Coffey, David 
Hammer, Daniel M. 
Levin, Terrance Grant 


Richard A. Duschl & 
Drew H. Gitomer 


Maria Araceli Ruiz- 
Primo, Erin Marie 
Furtak 


Bronwen Cowie & 
Beverley Bell 


Source 


Educational Measurement: 
Issues and Practice 


Assessment in Education: 
Principles, Policy & Practice 


Science Education 


Educational Assessment, 
Evaluation and Accountability 


Journal of Research in Sci- 
ence Teaching 


Educational Assessment 


Journal of Research in Sci- 
ence Teaching 


Assessment in Education: 
Principles, Policy & Practice 
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Citations anak Cluster 
5 23 1 
5 20 1 
5 14 1 
5 12 1 
4 22 ‘ 
4 21 ‘ 
4 21 1 
4 20 1 


The cluster analysis results indicated that the top 10 references by co-citation were frequently cited together 
(see Figure 5). The findings for this result produced two clusters, including the top ten references. The studies of 
Bell and Coiwe (2002), Bennet (2011), Black and William (1998), Black and William (2009), Coffey et al. (2011), Cowie 
and Bell (1999), Duschl and Gitomer (1997), Furtak (2012), (Heritage et al., 2009), and (Ruiz-Primo & Furtak, 2007) 
comprised the first cluster. The second cluster consisted of Bell and Coiwe (2001), Black and William (1998), Black 
and William (2009), Coffey et al. (2011), Hattie and Timperlay (2007), Shepard (2000), White and Frederiksen (1998) 
studies. 
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Figure 5 
Network Analysis of References by Co-Citation Analysis 
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Research Fronts in Formative Assessment 


Bibliographic coupling analysis is ideally suited for identifying research fronts because it offers two advan- 
tages (Boyack & Klavans, 2010). Firstly, it can pinpoint newly researched topics. Secondly, it can complement the 
findings of keyword co-occurrence analysis. The bibliometric coupling method is useful for identifying articles 
and researchers related to similar topics, but it does not evaluate the quality of the studies. In the analyses, the 
bibliographic coupling of documents was performed in a formative assessment to highlight the research front 
(themes). The results yielded four clusters. Accordingly, cluster 1 included Decristan et al. (2015), Genlott & Gron- 
lund (2016), Grob et al. (2017), Hondrich et al. (2016), Maier et al. (2016), and Prince et al. (2020). The studies in this 
cluster focused on the effects of formative assessment on students’ understanding and learning outcomes and 
analyzed teachers’ challenges. For example, Decristan et al. (2015) examined the effects of formative assessment 
on students’ science understanding. Genlott and Gronlund (2016) studied the effects of a “Write to Learn” (WTL) 
method using written real-time formative feedback on students’ learning outcomes. Hondrich et al. (2016) evalu- 
ated primary school science teachers’ implementation fidelity under two conditions. Grob et al. (2017) investigated 
teachers’ challenges when implementing formative assessment methods in inquiry-based science classrooms and 
analyzed their recommendations. Maier et al. (2016) developed computer-assisted formative assessments for an 
evolutionary adaptations instruction unit and reported their effects on studied variables. 

Cluster 2 consisted of Adams and Wieman (2011), Coffey et al. (2011), Cohen and Sasson (2016), Dong et al. 
(2009), and Shen and Linn (2011). In this cluster, the studies emphasized the importance of formative assessment 
in science teaching. For example, Adams and Wieman (2011) created an assessment test that measured the effec- 
tiveness of instruction in a specific area of science and used formative assessment techniques. Coffey et al. (2011) 
examined the state and development of formative assessment research. 

Cluster 3 included Archila et al. (2018), Furtak (2012), Furtak and Heredia (2014), and Furtak et al. (2016). 
Archila et al. (2018) studied the effects of a formative assessment-based strategy on university students’ argumen- 
tation. The studies of Furtak and her colleagues focused on teachers’ understanding of formative assessment in 
teaching processes. 

Cluster 4 included the studies of Buck et al. (2010), Offerdahl and Tomanek (2011), Tomanek et al. (2008), 
and Trauth-Nane and Buck (2011). These studies examined the processes of teachers and instructors regarding 
formative assessment. For example, an inquiry process with formative assessment was the focus of Buck et al’s 
(2010) action research. A case study was conducted by Offerdahl and Tomanek (2011) to explore the assessment 
strategies of three science instructors at a research university in the United States. Tomanek et al. (2008) sought to 
identify factors that influenced the reasoning of prospective and experienced secondary science teachers’ forma- 
tive assessment tasks for their students. In 2011, Trauth-Nane and Buck researched the effectiveness of reflective 
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practice in an action research study. Their study focused on incorporating formative assessment into middle school 
science teaching and learning. 


Figure 6 
Research Fronts in Formative Assessment 
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Hotspot Themes in Formative Assessment 


The frequency analysis revealed the ten most frequently occurring keywords associated with the formative 
assessment. These are formative assessment, students, science education, teaching, education, engineering edu- 
cation, curricula, stem, stem (science, technology, engineering, and mathematics), and e-learning (see Table 5). 


Table 5 
Ten Most Commonly Used Keywords 


Keyword Occurrences Total link strength 
Formative assessment 82 154 
Students 29 109 
Science education 21 46 
Teaching 20 80 
Education 19 73 
Engineering education 11 43 
Curricula 10 46 
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Keyword Occurrences Total link strength 
Stem 10 31 
Stem (science, technology, engineering and 
‘ae 8 33 
mathematics) 
E-learning 8 32 
Assessment 8 21 


In addition, themes associated with “hotspots” may emerge at various times and are distinct from research 
fronts. These themes that emerge from keyword co-occurrence analysis are displayed in four clusters and groups. 
The keywords are naturally clustered into four groups, as determined by cluster analysis (see Figure 7). Keywords 
belonging to the same cluster frequently appear within the same occurrence. Each cluster may reveal multiple 
research hotspots. Even though the keywords between clusters are distinct, different clusters may point to the 
same research objective in some of the research literature. Through an in-depth analysis of the keywords in each 
cluster, we can understand the nature of research hotspots on formative assessment. Publications containing 
Cluster 1 (purple) focused on students and assessment and contained curricula, education, education comput- 
ing, engineering education, students, and teaching keywords. Cluster 2 (light blue color, orange, and red) focused 
on STEM and included assessment, feedback, STEM, and STEM education keywords. Cluster 3 (red) was named 
e-learning and involved Computer aided instruction, e-learning, higher education, and learning systems. Cluster 
4 (green and orange) focused on formative assessment and comprised Formative assessment, science education, 
stem (science, technology, engineering, and mathematics), and teaching practices keywords. 


Figure 7 
Keywords in Co-Occurrence Analysis 
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Research Trends in Formative Assessment 


An analysis of the occurrence of keywords provides a diachronic overview of the evolution of research 
on formative assessment over the past fifteen years regarding major themes (see Figure 7). Figure 7 shows the 
most popular themes in formative assessment research over time. In 2015, topics such as curriculum, education, 
education computing, teaching, and engineering education appeared. Between 2015 and 106, the theme of 
science education appeared in the literature in formative assessment studies. Between 2016 and 2017, themes 
such as students appeared in the studies. In 2017, themes such as formative assessment, higher education, 
and teaching practices emerged in the studies. In 2018, the research included themes such as STEM and STEM 
education. In 2019, the topics found in the studies were STEM, e-learning, computer-aided instruction, learning 
systems, and feedback. 


Discussion 


This study aimed to examine the bibliometric results of articles on formative assessment in science educa- 
tion. For this aim, the SCOPUS-indexed publications on formative assessment in science education were analyzed 
to answer the research question in this study. Based on the criteria for inclusion and exclusion, 94 articles were 
examined. The results revealed an increasing publication trend on formative assessment in science education 
between 2015-2016 and 2020-2022. Accordingly, the number of publications on formative assessment has 
increased substantially. These results showed a research trend on formative assessment in science education 
literature. In particular, much of the research has been conducted in the last decade. It is unsurprising that much 
of the literature on formative assessment has been published in the last decade, given the strong connection 
between science and STEM education. 

The results also revealed that seven of ten institutions that contributed to the research on formative 
assessment were from the United States. The first three institutions are the United States and the University of 
Colorado Boulder, the University of Nebraska-Lincoln, and the University of California, Berkeley. In addition, nine 
of the top ten institutions are from developed countries except Colombia. After analyzing the data, it is clear that 
institutions in developed countries focused more on formative assessment in their research compared to those 
in developing countries. The top ten institutions show less diversity since seven are from the same country. This 
result also suggests that American universities have conducted more research on formative assessment. 

Another finding was revealed from the results regarding the number of research by countries. The re- 
sults demonstrated that The United States had the most publications (see Figure 2). Germany and Israel tied for 
second place. The Netherlands ranked third among the nations. Later, Colombia, Denmark, the United Kingdom, 
Greece, Spain, Turkey, and the United Kingdom contributed to the research regarding the number of publica- 
tions on formative assessment. In particular, the United States had the most publications. This result can explain 
researchers’ interest in formative assessment in American universities. This result is similar to that of Sudakova 
et al. (2022). They found articles from 20 countries on online formative assessment in a bibliometric study. In 
particular, their findings suggest that the United States is a leader in research on online formative assessment. 
They indicated that the top countries in online formative assessment are the U.S., the U.K., Australia, Spain, and 
Germany. Their results are partly similar to those of the findings in this research. 

The results regarding publication sources of formative assessment research showed that the most cited 
journals, the Journal of Research in Science Teaching (JRST), Science Education, and the International Journal 
of Science Education (IJSE) journals, had received more citations on formative assessment research than one 
hundred times. As scholars know, citations are a reliable way to evaluate the importance and quality of research. 
The articles with the most citations in a particular field significantly influence other researchers. Articles with 
over 100 citations can indicate that other scholars are building on the research of these authors to conduct more 
impactful studies. Results regarding the cluster analysis showed two clusters. The journals in the first cluster are 
JRST, Science Education, Journal of Science Teacher Education, Educational Researcher, Assessment in Education: 
Principles, Policy & Practice, Applied Measurement in Education, and Teaching and Teacher Education. The second 
cluster comprised the IJSE, Educational Research Review, and Computers & Education journals. One common 
feature among these journals is that well-known publishers have published them for over 30 years. In addition, 
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it can be thought that these most cited journals are the most prominent in science education literature. Another 
issue is that journals with high impact factors are typically regarded as more influential, of higher quality, and 
frequently cited. Therefore, scholars researching formative assessment could prefer to publish their findings in 
these prestigious journals to increase the impact of their research. 

Another finding is that results regarding most co-cited authors showed that Dylan Wiliam, Erin Marie Fur- 
tak, Paul Black, Maria Araceli Ruiz-Primo, and Richard J. Shavelson were cited more than fifty times. This finding 
is not surprising as these authors are probably the leading authors of formative assessment. The remaining 
five authors, Bronwen Cowie, Christine Harrison, Marcia C. Linn, Yue Yin, and Beverley Bell, were co-cited over 
thirty times. The results also show that six top ten authors are from the United States. Meanwhile, three authors 
come from New Zealand, and one is from the United Kingdom. This result also demonstrates that the most cited 
authors come from developed countries. Based on the results, it is likely that developed countries are heavily 
involved in advancing scientific education through scholarly research. This result is partially similar to the find- 
ings of Sudakova et al. (2022). 

Lastly, the results regarding the network analysis of keywords revealed that the ten most frequently oc- 
curring keywords were formative assessment, students, science education, teaching, education, engineering 
education, curricula, stem, stem (science, technology, engineering, and mathematics), and e-learning. As Chen 
et al. (2020) suggested, keyword analysis can help researchers identify relevant research trends, with the most 
frequently used keywords indicating the most researched topics. Much of the research has been conducted 
in the context of STEM education; the keywords STEM and STEM education have emerged as the central key- 
words in cluster analysis. This result also shows that formative assessment has received attention from science 
and STEM education researchers. This result is different from those of Sudakova et al. (2022). They found that 
between 2010 and 2020, the keyword “blended learning” rose to the top of the most popular topics in their 
bibliometric analysis regarding online formative assessment. In addition, their analysis revealed that between 
2011 and 2019, “formative assessment” and “assessment” were extremely common. After 2015, they found that 
authors frequently added “Moodle” to the keywords regarding formative assessment. 


Conclusion and Educational Implications 


Although the number of publications on formative assessment in science education has long been high, 
there is no bibliometric analysis or scientific mapping in the literature to understand research trends in formative 
assessment research. This research aimed to examine the bibliometric results of articles on formative assess- 
ment in science education. The results showed that the development of studies on formative assessment had 
reinforced an increasing publication trend in science education between 2015-2016 and 2020-2022. The five 
most influential authors are Dylan Wiliam, Erin Marie Furtak, Paul Black, Maria Araceli Ruiz-Primo, and Richard 
J. Shavelson. The five most influential journals are the JRST, Science Education, the IJSE, the Journal of Science 
Teacher Education, and Educational Researcher. The most influential countries from which research papers 
originate are the University of Colorado Boulder, the University of Nebraska-Lincoln, the University of California, 
Berkeley, the University of Missouri, the School of Natural Resources, and the University of the Andes. The result 
shows that academics from the United States have made considerable contributions to this field, and American 
universities employ the best researchers working on formative assessment. The findings have suggested that 
most publications and top authors in this field are from developed countries. This research emphasizes that 
the majority of studies on formative assessment are conducted in developed nations’ contexts. This finding 
was further supported by the top ten references, which showed that the most cited documents were also from 
developed nations. According to the study, formative assessment has been the subject of extensive research in 
numerous fields. It is now one of the most popular research topics in science education. As developed nations 
(e.g., the United States and Germany) receive more attention, many scholarly publications have been focused 
on the Western context. 

According to the keyword co-occurrence analysis, the ‘hotspot’ themes and potential research trends 
pointed out formative assessment, students, science education, teaching, engineering education, curricula, stem, 
stem (science, technology, engineering, and mathematics), and e-learning. Theoretically, this study confirms 
a research trend for formative assessment based on the bibliometric characteristics of the retrieved records. 
It reveals information about research on formative assessment. More specifically, more research on formative 
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assessment is essential to determine its impact on student and teacher outcomes and uncover existing chal- 
lenges and difficulties that can provide solutions for effectively teaching science and STEM subjects in schools. 

While the articles offer valuable insights for researchers in less developed countries, examining the state of 
research in developing countries is essential. It is recommended that future studies should focus on formative 
assessment research in other countries. It is worth noting that many scholars from developed nations contribute 
to establishing a knowledge base on formative assessment. We also encourage researchers and practitioners 
from developed and developing nations to collaborate on formative assessment, as our study has identified a 
research gap in this area. 

Although there has been an increasing interest in formative assessment within science education litera- 
ture, there has been limited knowledge of the current status of research using bibliometric analysis. To our 
knowledge, no research has examined the current state of formative assessment bibliometrically, except for one 
study. Therefore, the authors needed to conduct this present research to contribute to the existing knowledge 
by providing an overview of the current research in this field. 

Bibliometric research is useful for policymakers and educators to make informed decisions regarding 
education policy, curriculum development, and instructional practices. This research provides a piece of 
evidence-based information on the research status of formative assessment in science education. Researchers 
can use bibliometric analysis to gain insights into the field, evaluate research impact, identify research gaps, 
and contribute to evidence-based educational decision-making. This study analyzed studies from the SCOPUS 
database. However, future research should consider including data from other databases like ERIC, EBSCOhost, 
and Web of Science. Further research can focus on tracking the development of this field, research directions, 
and international cooperation networks between nations and organizations. Scholars can also study changes in 
research results on formative assessment in science education separately for students and teachers. To explore 
different aspects of research on formative assessment, future studies should conduct more bibliometric research 
to identify research trends. 


Limitations 


Although this study provides a comprehensive knowledge map of research on formative assessment in 
science education literature, some limitations should be noted. This study uncovered collaboration potential 
through bibliometric analysis and provided valuable results for formative assessment research. Although 
this study provides valuable insights into formative assessment studies, it is important to note that the 
search period may have limited the scope and depth of the included studies. One significant limitation of 
the study is that it only included research studies indexed by the Scopus database. It is possible that other 
valuable studies on formative assessment were not indexed by Scopus, making them inaccessible for this 
study. In addition, we were unable to review studies conducted in local languages. On the other hand, the 
analyses were performed using the keywords selected by the authors. Our search parameters, which include 
only SCOPUS-provided and English-language documents, may exclude relevant studies unavailable through 
SCOPUS or written in other languages. Future bibliometric studies should expand the search parameters to 
include additional studies. 

Researchers using other databases for this type of research using bibliometric analysis may reach different 
conclusions. Researchers can create a more thorough analysis by combining studies from multiple databases 
when conducting bibliometric studies. Depending on their interests, researchers interested in formative 
assessment may also conduct studies on “alternative assessment,’ “self-assessment,’ and “authentic assess- 
ment” in the educational context. Finally, additional research is required to conduct a more in-depth analysis. 
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Abstract. /n recent years, developing 
students’ knowledge integration of specific 
topics through effective teaching methods 
has been a concern in science education. 
This study explored an alternative teaching 
method, which aims to help lower 
secondary school students develop their 
knowledge structure about buoyancy 
towards a logically integrated conceptual 
framework through enhancing their 
understanding of the nature of buoyancy. 
Specifically, this study constructed a 
conceptual framework on buoyancy 

and aset of matching test questions to 
map out the three levels of knowledge 
integration of students, and designed and 
implemented a teaching experiment of 
promoting students’ understanding of the 
nature of buoyancy. Through quantitative 
and qualitative analysis, it was found that 
the knowledge integration development 
of the treatment group was significantly 
better than that of the control group. The 
results indicated that enhancing students’ 
understanding of the nature of buoyancy 
can effectively improve their knowledge 
integration of buoyancy. Accordingly, 
improving knowledge integration through 
enhancing students’ understanding of the 
nature of the specific scientific concept may 
be anew direction for future research. 
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IMPROVING LOWER 
SECONDARY SCHOOL 
STUDENTS’ KNOWLEDGE 
INTEGRATION OF BUOYANCY 
THROUGH ENHANCING THEIR 
UNDERSTANDING OF THE 
NATURE OF BUOYANCY 


Yi Zou, 
Xinyu Xue, Lizhen Jin, Xiao Huang, 
Yanbing Li 


Introduction 


In recent years, cultivating students’ ability to solve practical problems 
has been a main goal of science education (National Research Council, 2012; 
Nieminen et al., 2012). As for how to develop this ability, many studies have 
mentioned that it is necessary to help students establish reasonable connec- 
tions among related scientific concepts and contextual features, leading to an 
integrated knowledge structure, so as they can obtain more effective reason- 
ing pathways in solving problems (Dai et al., 2019; Liu et al., 2022). Therefore, 
it has been a concern in science education to seek effective teaching methods 
for improving students’ knowledge integration about specific topics (Bao & 
Fritchman, 2021; Scalise, 2012). This study explored an alternative teaching 
method, which aims to improve students’ knowledge integration about a 
specific concept topic through understanding the nature of the concept. 

The concept topic of this study is buoyancy. As for why the topic of 
buoyancy was chosen, it is related to the personal experience of the first 
author of this study. The first author once worked as a science teacher in a 
lower secondary school in Zhejiang Province, China. When he taught the topic 
of buoyancy, he found that it was generally difficult for students to construct 
an integrated knowledge structure about buoyancy. At the same time, there 
seems to be a deviation in the definition of buoyancy in general science teach- 
ing in China, which greatly hinders students from correctly understanding 
the nature of buoyancy, and this may be the main reason why students are 
unable to effectively construct an integrated knowledge structure about 
buoyancy. In other words, understanding the nature of buoyancy may be 
the key to realize an integrated knowledge structure about buoyancy. This 
study was a verification of this hypothesis. 
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The following were the two main aims of this study: (1) establish a conceptual framework and a set of match- 
ing test questions on the topic of buoyancy to map out students’ different levels of knowledge integration of 
buoyancy; (2) design and implement a teaching experiment of promoting students’ understanding of the nature 
of buoyancy, and verify its effect on students’ knowledge integration of buoyancy. 


Literature Review 
Knowledge Integration in Science Education 


In science education, there is acommonly accepted viewpoint that students’ problem-solving performance on 
a certain topic depends on the knowledge structure they have constructed (Gerace et al., 2001; Snyder, 2000). For 
expert students, they have already established a complete conceptual framework, which contains relevant knowl- 
edge elements such as ideas, conceptions, principles, equations, contextual variables, etc., and these elements are 
meaningfully connected through some reasoning pathways (Bao & Fritchman, 2021; Eylon & Reif, 1984). From this 
perspective, the reason why expert students excel in solving related problems is that the contextual characteristics 
will activate their conceptual framework, and then some reasoning pathways will emerge as the strategies to solve 
the problems (Dai et al., 2019; Xie et al., 2021). On the contrary, novice students’ knowledge structure is relatively 
fragmented. Although it also contains some knowledge elements, they are mechanically memorized, loose and 
unorganized (Eylon & Reif, 1984; Gerace et al., 2001). When novice students solve problems, they tend to directly 
match contextual characteristics with knowledge elements in memory, without a meaningful reasoning process, 
such as directly substituting contextual variables into relevant formulas for calculation. Once direct matching is 
not feasible, they have no alternative problem-solving strategies (Bao & Fritchman, 2021; Dai et al., 2019). 

Accordingly, the critical way to enhance students’ problem-solving ability lies in helping them construct a 
more integrated knowledge structure for the specific topic, and characterizing students’ knowledge structure with 
an ideal conceptual framework is the necessary groundwork. In this regard, it is worth noting that individual cogni- 
tive characteristics may vary, and each topic may involve diverse knowledge elements and reasoning pathways. 
This means that the ideal conceptual framework should not be unique, and the appropriateness should be the 
principle in its construction (Dai et al., 2019; Xu et al., 2020). 

Determining a central idea is the first and key step in the construction of a conceptual framework (Dai et al., 
2019; Kubsch et al., 2018). The central idea has strong capacity for attracting other knowledge elements, like an 
anchor point that can be connected with other knowledge elements through specific logical relationships, leading 
to an integrated conceptual framework (Xie et al., 2021; Xu et al., 2020). The established conceptual framework can 
not only guide the implementation of teaching activities, driving students to construct this conceptual framework, 
but also serve as a basis for designing assessment instruments, analyzing students’ development of this conceptual 
framework (Dai et al., 2019; Xie et al., 2021). 


Student Learning of Buoyancy 


The sinking and floating of objects in liquid is a very common life phenomenon for students, but buoyancy 
is a relatively difficult scientific concept for them (Radovanovic, 2012; Wagner et al., 2013). Relevant studies have 
found that students make many mistakes in learning buoyancy, some of which are caused by their daily experi- 
ences, such as the block of wood floats and the block of iron sinks because the latter is heavier, emerging objects 
are subject to buoyancy while sinking objects are not subject to buoyancy, and so on (Tomo, 2021; Wagner et al., 
2013). From these mistakes, it can be seen that students need to go beyond the relevant daily experience to cor- 
rectly understand buoyancy. 

This fact is consistent with some descriptions of the characteristics of understanding scientific concepts, that 
is, understanding scientific concepts is not as intuitive and self explanatory as understanding daily concepts, but 
often requires an abstract thinking process of metaphor and modeling (Asakle & Barak, 2020; White & Frederikson, 
2009). Unfortunately, the general teaching of buoyancy does not seem to be well aware of this key point. Taking the 
general teaching of buoyancy in China as an example, the definition of buoyancy is generally introduced from the 
perspective of daily experience, that is, “The force that the object immersed in the liquid is lifted vertically by the 
liquid is called buoyancy.’ And the teachers usually simply explain the definition of buoyancy based on students’ 
relevant daily experience, and then verify the existence of this vertical force through a simple experiment (Hang 
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a heavy object on the spring scale, and then immerse it in the liquid. It can be observed that the indication of the 
spring scale decreases.) (Gao et al., 2020; Shen et al., 2015). Next, the teachers will lead the students to learn other 
contents related to buoyancy. 

However, previous studies have shown that the general teaching like the one mentioned above leads to stu- 
dents being unable to understand the nature of buoyancy and establish reasonable relationships among buoyancy 
and relevant knowledge elements such as variables, principles, equations, and so on (Minogue & Borland, 2016; 
Paik et al., 2017). Students tend to mechanically rely on the matching patterns in memory when solving buoyancy 
related problems. As a result, many students perform well on simple and familiar problems, but poorly on complex 
and unfamiliar problems (Kim & Paik, 2021; Young & Meredith, 2017). 

Recognizing these shortcomings, researchers began to explore corresponding solutions. Some studies 
pointed out that the learning of buoyancy involves too many knowledge elements that students cannot handle 
simultaneously, so we need to help students find a certain effective way to integrate them (Minogue & Borland, 
2016; Paik et al., 2017). Further studies on this topic have confirmed the potential significance and effectiveness of 
understanding the nature of buoyancy in integrating knowledge elements. The explanations of buoyancy related 
phenomena and the solutions to buoyancy related problems are truly correct only on the basis of considering the 
nature of buoyancy, which refers to the fact that buoyancy is an effect force exerted on the surfaces of an immersed 
object, resulting from the liquid gravity (Kim & Paik, 2021; Lima & Monteiro, 2013). 


The Conceptual Framework and Assessment Questions for Mapping Out Students’ Knowledge Integra- 
tion in Learning of Buoyancy 


The Conceptual Framework 


Based on the above discussions, the nature of buoyancy was selected as the central idea. Then, we invited 3 
professors of science education and 2 well-experienced science teachers to form an expert group to construct an 
appropriate conceptual framework through consultation. 

Firstly, the expert group also believed that the nature of buoyancy is that buoyancy is an effect force exerted 
on the surfaces of an immersed object, resulting from the gravity of a liquid. Therefore, the accurate definition 
of buoyancy should be “The net force exerted by liquid pressures on the surfaces of an immersed object” (Kim & 
Paik, 2021; Lima & Monteiro, 2013). Chinese lower secondary school students are taking an elementary science 
curriculum, so this study only considered the net force in the vertical direction. As for the motion state of an im- 
mersed object in the vertical direction, it is determined by the buoyancy together with other forces acting on it 
vertically. Accordingly, the expert group pointed out that, on the basis of understanding the nature of buoyancy, 
students may perform the forces analysis of the immersed object according to its motion state, thus obtaining 
some equations for calculating buoyancy. In general, there are five motion states when an object immersed ina 
liquid partially or completely: emerged, suspended, sunk, emerging, sinking. The emerged, suspended and sunk 
objects are all in equilibrium vertically, with the difference that the buoyancy on the emerged or suspended ob- 
ject is equal to its gravity, while the sum of buoyancy and support force on the sunk object is equal to its gravity 
according to the forces synthesis principle and two forces balance principle. Accordingly, the equations F,, = mg 
and F,, = mg — F, are available for buoyancy related calculations (The F, in the equation refers to the containers 
support force for the object. And in science course of Chinese lower secondary school, the space between the 
bottom of the container and the object is assumed to be filled with liquid.). Conversely, the emerging or sinking 
object is not in equilibrium, for which the principle of two forces balance no longer applies. The basis for solving 
relevant problems is the Archimedes principle, that is, the buoyancy of an immersed object is numerically equal 
to the displaced liquid gravity, expressed by the equation F,, = pgv. The Archimedes principle is a reflection of 
the nature of buoyancy on the level of mathematical relationship. In the absence of an immersed object, the cor- 
responding volume would be occupied by a liquid whose gravity would be supported by the rest of the liquid 
(Kubsch et al., 2018; Young & Meredith, 2017). From this point of view, the Archimedes principle applies to the 
immersed objects in all five motion states mentioned above. 
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Conceptual Framework of Buoyancy 
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Based on the above discussion, this study established a conceptual framework of buoyancy, which can ex- 
press a logically integrated knowledge structure of students in learning of buoyancy (As shown in Figure 1). And 
according to the representative knowledge elements and reasoning pathways of the conceptual framework, the 
expert group believes that students’ knowledge integration of buoyancy can be roughly divided into three levels: 

Low level: The best situation for students at this level is that they have constructed a mechanical connec- 
tion between the calculation equations and the contextual variables (See the red dotted box in Figure 1). 
For example, when the mass of the immersed object, the acceleration of gravity, and the support force 
are directly given, students can mechanically substitute these variables into the equation F,, = mg — F, 
to calculate the buoyancy. However, when any of these three variables is not given in the problem, they 
will have no solution strategy except guessing. 

Intermediate level: Students at intermediate level can integrate more knowledge elements (See the green 
dotted box in Figure 1). The difference between them and the low level students is that they can under- 
stand the reasoning pathways among the calculation equations, the motion states, and the related ideas 
& principles. For example, when an object is emerging, these students can infer that other equations are 
no longer valid and turn to the Archimedes principle. If the required variables are not provided directly, 
they can obtain these variables through logical reasoning from other relevant information. However, 
in the case of the object sticking to the container, these students will be baffled because they have not 
understood the nature of buoyancy. 

High level: Students at this level have understood the nature of buoyancy, and driven by this under- 
standing basis, they have established a well-integrated conceptual framework of buoyancy (See the blue 
dotted box in Figure 1). When solving buoyancy related problems, no matter which contextual variables 
are given, these students have a great chance of success to come up with solutions based on complex 
connections between knowledge elements. 


The Assessment Questions 
According to the constructed conceptual framework and the division of students’ knowledge integration, a 


test consisting of 12 single-choice questions (1/4) was designed to assess students’ different knowledge integra- 
tion of buoyancy (The single-choice question (1/4) is one of the most common assessment methods in Chinese 
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lower secondary schools). These 12 questions were all derived from the questions that students often encounter 
in their daily study, removing the complex calculations and the irrelevant information. According to the degree of 
knowledge integration, these questions can be divided into three sets: single set, multilink set, and integrated set. 

Q3, Q5, and Q8 belong to the single set, which only involves the application of the contextual variables and 
the calculation equations. To obtain the correct answer, students only need to insert the given variables into the 
corresponding equations. For example, Q3 provides all the required variables required for the equation F, = pgv. 

Q1, Q4, Q10, and Q11 belong to the multilink set, which also need to be solved by using one of the three 
equations. However, the variables of the equations are not directly given, but can be obtained by connecting the 
relevant ideas & principles and the correct reasoning pathways. For example, Q4 gives the motion state of a com- 
plex system whose total gravity can be calculated indirectly and then can be solved with the equation F,, = mg. 

Q2, Q6, Q7, Q9, and Q12 belong to the integrated set, which can be solved when students have understood 
the nature of buoyancy and formed an integrated conceptual framework. The situational characteristics of these 
questions are relatively complex and unfamiliar to students, for example, the situation of Q6 is that the object is 
trapped in mud. Understanding the nature of buoyancy is a prerequisite for solving these questions, and then the 
corresponding reasoning pathways selected and applied on this basis can be fitting. 

It was expected that all students can answer most of single questions after learning the buoyancy courses, 
unless they use the wrong equations or make calculation mistakes. Most intermediate and high level students can 
answer most of multilink questions. Only the high level students can maintain excellent performance in integrated 
questions. 

The reliability of this test has been verified in another study (Zou et al., 2022). This study was used to analyze 
the effect of understanding the nature of buoyancy on corresponding knowledge integration after the teaching 
experiment. The detailed test questions are attached in the attachment. 


Research Methodology 


Background 

On the basis of constructing the conceptual framework and assessment questions for mapping out students’ 
knowledge integration in learning of buoyancy, this study designed intervention lessons to enhance students 
understanding of the nature of buoyancy. In September 2022, a total of 163 students participated in the teaching 
experiment, of which 80 students received intervention lessons as the treatment group, and the remaining 83 
students received general lessons as the control group. Before and after the buoyancy lessons, all 163 students 
participated in the pre-test and post-test, and some of them continued to participate in further interviews. Then, 
various quantitative and qualitative analysis methods were applied to explore the effect of the intervention lessons 
on students’ knowledge integration. 


O 


Participants 


The participants consisted of 163 8th grade students (14-15 years) in four classes, with 79 boys and 84 girls, 
who were all from a lower secondary school located in a midsize city Jinhua, Zhejiang) in China. Before learn- 
ing buoyancy, they had learned the relevant concepts and principles involved in the conceptual framework. 83 
students in two classes were randomly selected as the control group to receive 4 general lessons of buoyancy, 
and 80 students in the other two classes served as the treatment group to receive 4 newly designed lessons that 
emphasize understanding the nature of buoyancy. 


Design and Implementation of the Intervention Lessons 
The general lessons for the control group students. The general teaching of buoyancy in Chinese lower 
secondary schools consisted of four 45-minute lessons. In the initial lesson, with the teacher's help, students per- 


ceived the existence of buoyancy from the perspective of daily experience and understood that the upward force 
produced by water or other liquids on an immersed object is called buoyancy. 


1124 


ASS https://doi.org/10.33225/jbse/23.22.1120 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


ISSN 1648-3898 /Print/ IMPROVING LOWER SECONDARY SCHOOL STUDENTS’ KNOWLEDGE INTEGRATION OF 
BUOYANCY THROUGH ENHANCING THEIR UNDERSTANDING OF THE NATURE OF BUOYANCY 
ISSN 2538-71 38 /Online/ (PP.1120-1136) 


In the second lesson, under the guidance of the teacher, students recognized the five possible motion states 
of immersed objects and analyzed the forces on objects in different motion states, and then derived two equations, 
F,, = mg,and F,, = mg — F, according to the three different equilibrium states (emerged, suspended, sunk). 

In the third lesson, students experimentally investigated the relationship between the magnitude of buoy- 
ancy and the amount of the displaced water, and generalized their conclusions to other liquids, so as to get the 
Archimedes principle and the corresponding equation F,, = pgv. 

In the last lesson, students were trained in real and typical question situations such as densitometers, ships, 
and submarines, reinforcing mechanical connections between various variables and above three equations. 

The 83 students of the control group attended the four general buoyancy lessons. In general, these lessons 
spent no time on explaining the nature of buoyancy, but focused on the derivation of the three equations and the 
application of them in specific situations. 

The intervention lessons for the students in the treatment group. The 80 students of the treatment group 
attended 4 newly designed 45-minute lessons that emphasize understanding the nature of buoyancy. In the initial 
lesson, the teacher guided students to understand the nature of buoyancy through a newly designed experiment, 
and come up with the improved definition of buoyancy. The experiment mainly was newly designed based on the 
idea of visualizing the nature of buoyancy (Amedeo et al, 2015; Moreira et al., 2013). 


Figure 2 
The Newly Designed Experiment 
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Firstly, the teacher prepared a cube device as shown in Figure 2(a) and a water container with a double-layer 
sink as shown in Figure 2(b). Then, the teacher explained to the students as follows: (1) Each surface of the cube 
device has a prototype rubber membrane to show the pressure effect of the liquid. (2) And the bottom of the in- 
ner layer of the container has a square gap, when the cube device is placed on the inner layer of the container, the 
pressure of the water of the outer layer will act on the rubber membrane at the bottom of the cube device through 
this gap. (3) With the increase of the water of the outer layer, the pressure will become bigger and bigger, and the 
deformation of the rubber membrane on the bottom of the cube device will become larger and larger until the 
cube device appears to float up. 

Secondly, the teacher put the cube device on the inner layer of the container, and then added water to the 
inner layer until the water exceeded the upper surface of the cube device for a certain distance. In this process, it 
can be observed that the rubber membrane on the left and right surfaces of the cube device first appeared larger 
and larger deformation. Since the water exceeded the upper surface of the cube device, the rubber membrane 
on the upper surface of the cube device also began to deform, and the degree of deformation was increasing. 
Meanwhile, the rubber membrane on the lower surface of the cube device has never been deformed. 

Thirdly, the teacher started to add water to the outer layer of the container. Since the water reached the 
gap, the rubber membrane on the bottom of the cube device began to deform, and the degree of deformation 
increased with the increase of the outer water until the cube device appeared to float up (Figure 2(c), 2(d), 2(e)). 

Finally, in combination with the above experimental process, the teacher guided the students to analyze the 
forces on the cube device as shown in Figure 2(f) to help students understand the nature and source of buoyancy, 
and defined buoyancy as “The net force of an immersed object exerted by liquid pressures on the surfaces.” 

The contents of the next three lessons were similar to those of the control group, but in every lesson, the 
teacher highlighted the connection between the contents and the nature of buoyancy. 

The general lessons and the intervention lessons are summarized as shown in Figure 3. All lessons were 
taught within one week by an outstanding science teacher recognized by the school, who had not taught these 
students before. 


Figure 3 
The General Lessons and the Intervention Lessons 


Guide students to perceive the existence of Guide students to understand the nature of 
buoyancy from the perspective of buoyancy through a newly designed experiment 
daily experience and obtain the improved definition of buoyancy 


Analyze the forces on objects in different motion 
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connection to the nature of buoyancy 


Analyze the forces on objects in different motion 
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the nature of buoyancy 


Introduce the Archimedes principle through a 
related experiment and get the equation 
Fy = pgv 


Apply the three equations to solve problems in 
different situations; highlight the connection to 
the nature of buoyancy 


Apply the three equations to solve problems 
in different situations 


(a) general lessons (b) intervention lessons 
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Data Collection and Analysis 


Both control and treatment groups took the pre-test and post-test before and after the lessons. Each test was 
completed in 30 minutes. Students’ pre-test and post-test performances were compared by two-way ANOVA, and 
further explored by T-test, so that the effectiveness of the intervention lessons can be judged. The score distribution 
for the different question sets was analyzed to reveal the difference of the development characteristics of students’ 
knowledge structure. At the same time, exploratory factor analysis (EFA) was performed on the post-test data to 
explore the impact of the intervention lessons on students’ knowledge integration. 

Additionally, a random sample of 30 students from each group participated in the think-out-loud interviews 
to illustrate their reasoning process of problem solving, so as to reveal their knowledge structure in greater depth 
after receiving the lessons. The interview of each participant took approximately 20 minutes and was recorded. 
The authors transcribed and sorted the same interview recordings into a preliminary interview text respectively, 
and finally formed a formal interview text for analysis after full discussion. To avoid the subjectivity of researchers 
affecting the objectivity of the analysis process, the authors analyzed the same interview data respectively, and 
explored different opinions through discussion and negotiation until the analysis results achieved refinement and 
agreement. 


Research Results 
Quantitative Analysis on Students’ Knowledge Integration Development 


The overview of the pre-test and post-test performances of the parallel groups is given in Figure 4. The pre- 
test data points of the two groups are almost coincident, which means these students were statistically identical 
in terms of conceptual understanding level ahead of the lessons. This is further confirmed by the results of ANOVA 
[F (2, 483) =.151, p = .860], which illustrate that there was no statistically significant interaction between different 
groups and question sets on the initial diagnostic test. 

As shown in Figure 4(a), the overall performances of the two groups of students have improved after the 
implementation of the buoyancy lessons, but the degree of improvement is significantly different [t, .. nent (79) = 
17.091, p < .001,d=1.910;t,,,., (82) = 9.262, p < .001, d= 1.016]. This demonstrates that the intervention lessons 
had a more positive effect on conceptual learning of buoyancy than the general lessons. In addition, the post- 
test ANOVA reveals significant interaction [F (2,483) = 5.269, p = .005], with main effects for both question sets [F 
(12,483) = 43.026, p < .001] and teaching conditions [F (1,483) = 52.758, p < .001]. Accordingly, the intervention 
lessons had different degrees of effects on students with regard to different question sets. As shown in Figure 4(b) 
(c) (d), for the integrated set, the advancement of the treatment group students’ performances is far superior to 
that of the control group [t treatment (29) = 15.411, p < 001, d = 1.722; t ...., (82) = 7.057, p < .001, d = 0.774]. With 
regard to the multilink set, the difference between the two groups is narrower [t treatment (72) = 12.888, p < .001, d= 
1.441; t .. ..(82) = 6.611, p < .001, d= 0.7256]. And the difference is further narrower for the single set [t ,,..ment (79) 
= 11.338, p < .001,d = 1.267; t .._..,, (82) = 6.546, p < .001, d = 0.7185]. These results manifest that the intervention 
lessons can help students understand the nature of buoyancy and form an integrated knowledge structure more 
effectively because, with the upgrading of the question set, students were required more to understand the nature 
of buoyancy and establish an integrated knowledge structure. 
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Pre-post Testing Performances of Students From the Two Groups 
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To explore the difference of the development characteristics of students’ knowledge structure about buoyancy, 
the distribution of scores for the different question sets was analyzed (see Figure 5). The students of the control 
group and the treatment group were further assigned to five crowds respectively according to their ranking of 
total scores in the post-test, each containing 20% of the group sample. As shown in Figure 5, the development 
trend of the scores between the control group students and the treatment students is similar. Specifically, for both 
groups, students’ scores on the integrated set are lower than the scores on the simple set and the multilink set for 
all crowds, except for those who get full marks. And for both groups, with the increase of students’ total scores, their 
scores on any question set also increase, and the trend of their performance gaps between any two question sets 
is similar. These results reveal a similar general progression in the two group students’ development of knowledge 
structure about buoyancy. However, the growth rate of the treatment group students’ scores on each question 
set is significantly faster than that of the control group, indicating that the intervention lessons had a statistically 
significant effect on the development of students’ knowledge structure in learning of buoyancy. 
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Figure 5 
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The results of exploratory factor analysis (EFA) are plotted in Figure 6, providing detailed information for 
further analysis of students’ knowledge structure after the implementation of the buoyancy lessons. For the treat- 
ment group, the first eigenvalue explains nearly half (47.101%) of the variance, indicating that the single factor can 
explain students’ performances on different questions (See Figure 6(a) for the factor loading of the questions). This 
means that these students have established a relatively integrated knowledge structure about buoyancy on the 
whole. For the control group, three common factors with eigenvalues greater than 1 are extracted, which explain 
21%, 20%, and 17% of the total variance, respectively, and their cumulative variance contribution is 58%. As shown 
in Figure 6(b), the plot of factor loading indicates that the three factors exactly represent the above-mentioned 
three question sets, factors 1, 2, 3 correspond to single set, multilink set, integrated set, respectively. And there is 

little correlation between the single set and multilink set (.412), between the single set and integrated set (.210) 
and between the multilink set and integrated set (.221). It can also be seen that control group students’ responses 
on the single questions and the multilink questions have a moderate correlation (.412), but their responses on the 
single questions (.210) and the multilink questions (.221) are less connected to the integrated questions. These 
results represent that the knowledge structure of control group students was relatively unintegrated on the whole 
and lacked an understanding of the nature of buoyancy. Comparing the analysis results of the two groups, the 
knowledge integration of treatment group students was significantly better than that of control group students. 
It can be concluded that the intervention lessons of emphasizing understanding the nature of buoyancy had a 


statistically significant effect on students’ knowledge integration of buoyancy. 
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Factor Loadings for EFA of Students’ Post-test Data 
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Qualitative Analysis on Students’ Reasoning Process 


Among 163 students, 60 students were randomly selected as the subjects of think-out-loud interviews to ex- 
ternalize the thought process when solving problems. Through the analysis of the interview results, their knowledge 
integration about buoyancy can be roughly divided into three levels, which is consistent with the previous division. 

Low Level: Students at this level can only mechanically link the equations in memory to the variables in 
context without a meaningful reasoning process. In other words, these students’ knowledge structure was at a 
preliminary level. They neither understood the nature of buoyancy nor knew the relationships among relevant 
ideas & principles. For instance, student 1 and student 2 scored well in the single questions, but they performed 
poorly on the multilink questions and integrated questions. 


Student 1: (For question 2) “The question gives the displaced liquid volume and the object mass. This confused me. 
Should | use the equation F;, = pgu or the equation F;, = mg? Then, | used F, = pgv randomly.” 

Student 2: (For question 12) “In question 12, the object is stuck to the bottom of the container, so | wanted to use 
F,, = mg — &.. But | was surprised to find that the given variables were inconsistent to this equation. Finally, | had to 
guess one option.” 


As student 1 mentioned, he tried to substitute the known variables directly into the equation F,, = pgv and 
F,, = mg, but either equation would fail because there is no buoyancy in outer space in the absence of gravity. 
Similarly, student 2 selected the equation F,, = mg — F, based on the motion state of the object, but the infor- 
mation given in the question is incomplete for the equation. Actually, since the object is stuck to the container’s 
bottom, that is, there is no liquid between the bottom of the container and the object, the object only receives a 
downward force from the liquid according to the nature of buoyancy. It can be seen from the responses of these 
two students that their knowledge structure was less integrated, which only contained the contextual variables, 
the relevant equations and the simple relationship between them. As a result, the direct pattern matching became 
the solitary problem-solving strategy for these students. 

Intermediate Level: These students can not only establish the relationships between contextual variables and 
the equations, but also make connections to related ideas & principles. However, students at this level still cannot 
effectively understand and apply the nature of buoyancy. 


Student 3: (For question 6) “The object is trapped in the mud at the container’s bottom, the equation F,, = mg — F, 
should be used. However, the buoyancy seems to be downward here, it’s impossible. | was confused about this question. 
” (For question 2) “I tried to solve this question by applying equation F;, = mg or F, = pgy, but | found that the two 
answers were different, | was so confused.” 


Student 4: (For question 9) “At the beginning, this object is stationary at the container’s bottom, F,, = mg — F,, so 
options A and B are incorrect. And then, the object is emerging, F,, + mg, So option Cis wrong. It’s D.” (A is ati the 
correct answer.) 


1130 


Ass https://doi.org/10.33225/jbse/23.22.1120 


Journal of Baltic Science Education, Vol. 22, No. 6, 2023 


ISSN 1648-3898 /Print/ IMPROVING LOWER SECONDARY SCHOOL STUDENTS’ KNOWLEDGE INTEGRATION OF 
BUOYANCY THROUGH ENHANCING THEIR UNDERSTANDING OF THE NATURE OF BUOYANCY 
ISSN 2538-7138 /oniines (pp.1120-1136) 


Student 3 and student 4 were able to recall the related calculation equations and obtain the required variables 
through connecting the corresponding analysis processes, which drove them to solve most of the single questions 
and the multilink questions smoothly. However, the intermediate level students’ accuracy in integrated questions 
was still low due to the lack of understanding of the nature of buoyancy. For question 6, since the object is sunk 
into the mud, it is only necessary to consider the downward pressure from the liquid on the object's upper surface. 
For question 9, it is also necessary to analyze the upward and downward pressure on the object respectively to 
get the answer. Therefore, the knowledge structure of these students at intermediate level should establish more 
logical connections around the nature of buoyancy to get more efficient solutions to problems. In terms of this, 
these students still had the characteristics of pattern matching when solving problems, but they have mastered 
more relevant patterns than the low level students. 

High Level: Students at this level have developed a deep understanding of the nature of buoyancy and a 
highly integrated knowledge structure. In other words, in the knowledge structure of these students, the contextual 
variables, the calculation equations, the related ideas & principles, and the central idea, were not isolated from 
each other, but logically connected. Therefore, these students were able to answer almost all questions correctly. 


Student 5: (For question 2) “The source of buoyancy is the liquid gravity, but the liquids in outer space are not af- 
fected by gravity, so the correct answer of this question is zero, because the g = 0.” 

Student 6: (For question 7) “The buoyancy is caused by the liquid gravity, and it is equal to the net force on the 
object's upper and lower surfaces. The object is in equilibrium, so the answer can be obtained through analyzing the 
forces on the object. After calculation, | got option D.” 


It can be clearly seen that these students considered the nature of buoyancy first when solving the related 
problems, and then comprehensively applied calculation equations, contextual variables, and related ideas & 
principles to solve the problem. As a result, these students performed better on all questions, especially on the 
integrated questions. 

The number and percentage at each level of the 60 interviewed students are shown in Table 1. This further 
corroborates the findings of quantitative analysis: Compared with the general lessons, the newly designed inter- 
vention lessons can help students understand the nature of buoyancy and establish an integrated knowledge 
structure more effectively. 


Table 1 
The Number and Percentage at Each Level of the 60 Interviewed Students 


Knowledge integration level (N(%)) 


Group Total 
Low Intermediate High 
Treatment Group 30 4(13.4) 13(43.3) 13(43.3) 
Control Group 30 13(43.3) 9(30.0) 8(26.7) 


Discussion 


This study developed an alternative teaching method that aims to improve lower secondary school students’ 
understanding of the nature of buoyancy and help them develop their knowledge structure of buoyancy into a 
logically integrated conceptual framework. Above all, it should be pointed out objectively that there are still a few 
limitations with this study. Firstly, this study was conducted in China, so the general applicability of the research 
findings to a broader context needs to be further verified. Secondly, the sample size of this study is relatively small, 
and the confidence of the outcomes needs to be further improved through a larger sample size in future studies. 
Despite these limitations, this study has made some significant contributions, which are discussed as follows: 
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Buoyancy is a Difficult Concept to Understand for Students 


Consistent with the findings of previous studies, the findings of this study also showed that buoyancy is 
indeed a difficult concept to understand for students (Kim & Paik, 2021; Wagner et al., 2013; Radovanovic, 2012). 
Although students’ performances on different question sets have improved significantly after the implementation 
of the buoyancy lessons, the degrees of improvement are not ideal. For the students of the control group, their 
knowledge structure is fragmented as a whole, and more than 70% of them are still at the low level or the inter- 
mediate level of knowledge integration, who barely understand the nature of buoyancy. For the students of the 
treatment group, their knowledge structure is integrated as a whole, but about 43% of them have not reached a 
high level of knowledge integration, and need to further deepen their understanding of the nature of buoyancy. 
Meanwhile, it can be seen from the students’ specific descriptions of their reasoning processes that most students 
rely on pattern matching to solve problems, which leads that they can only deal with relatively typical and famil- 
iar problems. Therefore, targeted cognitive diagnosis and teaching methods are needed to help students better 
understand buoyancy. 


The General Teaching of Buoyancy is not Conducive to Students’ Knowledge Integration 


In the general teaching of buoyancy in China, the definition of buoyancy does not really reveal the nature of 
buoyancy. What's more, all the general buoyancy lessons mainly focus on helping students get familiar with the 
solution patterns of buoyancy related problems, rather than helping students establish the integrated knowledge 
structure. As in other research, under the influence of the general teaching, students in this study cannot effectively 
construct logical correlations and reasoning pathways among buoyancy related knowledge elements, and they 
depend on mechanical matching rather than meaningful reasoning when faced with relevant problems (Minogue 
& Borland, 2016; Tomo, 2021). Accordingly, the general teaching of buoyancy is only conducive to students’ familiar- 
ity with relevant problem situations and problem-solving patterns, but is not conducive to students’ knowledge 
integration, leading them to often encounter troubles in the face of complex and unfamiliar contexts. In view of 
this, it is necessary to explore some new teaching approaches that contribute to knowledge integration. 


The Teaching of Emphasizing the Nature of Buoyancy is Effective for Students’ Knowledge Integration 


This study constructed a conceptual framework of buoyancy through a series of logical relationships to di- 
agnose students’ knowledge integration of buoyancy. At the same time, this study designed and implemented 4 
new buoyancy lessons of emphasizing understanding the nature of buoyancy. The results of data analysis showed 
that understanding the nature has a significant effect on students’ forming an integrated knowledge structure of 
buoyancy. Although this conclusion is consistent with previous relevant research studies, this study has made new 
progress in presenting students’ cognitive schematics and teaching method design of helping students understand 
the nature of buoyancy (Gao et al., 2020; Paik et al., 2017; Zou et al., 2022). Therefore, more teaching methods 
that can help students understand the nature of buoyancy, as well as more logical conceptual frameworks about 
buoyancy, should be sought to help students better understand the nature of buoyancy and develop a more 
comprehensive knowledge integration of buoyancy. 


Conclusions and Implications 


Overall, this study has confirmed that the method of constructing a conceptual framework on buoyancy and 
a set of matching test questions is effective in mapping out lower-secondary school students’ different conceptual 
understanding in learning buoyancy. Meanwhile, this study designed and implemented an effective teaching 
method to develop students’ knowledge integration of buoyancy through enhancing their understanding of the 
nature of buoyancy. The experiment result indicated that the knowledge integration development of the treat- 
ment group students was significantly better than that of the control group students, and the treatment group 
had significantly more students who reached the intermediate and high levels than the control group. 

These results verified the effectiveness of understanding the nature of buoyancy in improving knowledge 
integration about buoyancy, which may bring a new approach to knowledge integration research in science edu- 
cation, that is, to enhance students’ knowledge integration about a specific concept topic by guiding students 
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to understand the nature of the concept. Of course, the adaptability of this approach needs to be further tested 
through subsequent research studies on more topics. On the other hand, for the research and practice of buoy- 
ancy teaching in China, this study is absolutely groundbreaking. Specifically, this study found that Chinese general 
teaching of buoyancy has a biased definition of buoyancy and the characteristics of attaching importance to the 
problem-solving pattern training, and provided a set of detailed reference strategies for targeted improvement. 
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Appendix 
The Assessment Questions Used in This Study 


(Tips: if gravity acceleration and water density are involved in the question, g is taken as 10N / kg, and water 
density is taken as 1.0 x 10 3 kg/m?; and in all cases that involve the object is sunk, unless otherwise specified, it is 
considered that the space between the bottom of the object and the bottom of the container is filled with liquid.) 


1.When an object is placed on the desktop, the supporting force of the desktop to it is 40N. 
When the object is put into the water, it is sunk and receives 30N supporting force from the 
bottom of the container. Please calculate the buoyancy of this object in the water. ( B ) 
ASN B.10 N C.30N D.40 N 


2. In outer space, if we put a small ball with a volume of 3 x10°m? and a mass of 2g into a cup 
with water, it would be suspended. What is the buoyancy of this small ball in the water? (A) 
AON B.0.01 N C.0.02 N D.0.03 N 


3. If we gently put an object into a cup full with water, 80 ml of water would overflow from 
the cup. Please calculate the buoyancy of this object in the water. (A ) 
A.0.8N B.1.8N C.2.8 N D.3.8 N 


4. As shown in the figure, there is an appropriate amount of water in the container. If we 
place the wooden block A with a gravity of 5N in the container and press the wooden block 
B with a gravity of 3N on wooden block A, wooden block A would emerge on the water with 
wooden block B. What is the buoyancy of wooden block A in the water at this time? ( C) 
A3N B.5 N C.8N D.10 N 
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5. James puts a small ball with a mass of 0.04kg into a container with water, and the state of 
it in the water emerges. Please calculate the buoyancy of this ball in the water. ( D ) 
A.0.1N B.0.2 N C.0.3 N D.0.4N 


6. As shown in the figure, the object with a volume of 600cm? and a gravity of 3N is sunk into 
the mud at the bottom of the container. It is known that the volume of the object trapped in 
the mud is 200cm?, the total embedding force of the mud on it is 7 N, and the force exerted 
by the water pressures on the upper surface of the object is 5N. If we want to salvage the 
object out of the mud, how much force is needed at least? ( D ) 

AIN B.10 N C.11N D.15N 


7. As shown in the figure, the hemispherical object with a volume of 700 cm? and a gravity 
of 4N is stationary at the bottom of a container with water. If we know that the force exerted 
by the water pressures on the upper surface of the object is 5N. What is the force exerted by 
the water pressures on the lower surface (spherical part) of the object? (D) 

A.2N B.AN C.6N D.8N 


8. The gravity of the object is 7N. When we put the object into the water, it would sink and 
receive 6N supporting force from the bottom of the container. Please calculate the buoyancy 
of this object in the water. ( A ) 

A.1N B.6 N C.7N D.13N 


9. Put the mouth of a lidless and bottomless beverage bottle facing downward, put the table 
tennis ball (the diameter is slightly larger than the diameter of the bottle mouth) into the bottle 
and inject water. You can see a small amount of water flowing out of the bottle’s mouth. At this 
time, the table tennis ball is stationary still, and then block the bottle mouth with your hand. 
After a while, the table tennis ball will float up. Which of the following analysis is correct? (A) 
A. the direction of the net force of liquid on the table tennis ball is vertical 
downward when it is stationary. 
B. The supporting force of the table tennis ball in the figure 

is balanced with the gravity when it is stationary. 
C. In the process of table tennis emerging, the buoyancy 
it receives is equal to the gravity it receives. 
D. In the process of table tennis emerging, the buoyancy 
it receives remains unchanged. 


10. As shown in the figure, the object is composed of a balloon and a stone (the stone is hung 
with the string under the balloon), and the object is suspended in the water. The mass of the 
stone is 0.4kg, the mass of the balloon is 0.1kg (including the air mass inside the balloon)and 
the mass of the string is 0.05kg. Please calculate the buoyancy of this object in the water. (A) 
A5.5N B.SN CAN D.1N 
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11. James puts the cuboid block with a total volume of 180 cm?into a container with water. 
When the block is stationary, the state of it in the water is emerged, and its volume above the 
water surface is 80cm?. What is the buoyancy of the cuboid block in the water? (B ) 

A.0.8 N B.1N C.1.8N D.2N 


12. The cuboid block with a volume of 600cm? and a gravity of 3N is stuck to the bottom of 
the container, and there is no gap between the block bottom and the bottom of the pool. It 
is known that the viscous force on the object is 5N and the downward pressure on the upper 
surface of the object is 4N. If we want to salvage the object from the water, how much force 
is required at least? (B) 

A.18N B.12N C6N D.3N 
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